-- Logs begin at Fri 2022-07-15 13:20:04 UTC, end at Fri 2022-07-15 14:21:24 UTC. -- Jul 15 13:45:42.950697 dv-cinder-850002-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 15 13:45:42.960206 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 15 13:45:42 2022] *** Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: nodename: dv-cinder-850002-1-iscsi Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: machine: x86_64 Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: clock source: unix Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: pcre jit disabled Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: detected number of CPU cores: 4 Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: current working directory: / Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: detected binary path: /usr/bin/uwsgi-core Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: your processes number limit is 47680 Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: your memory page size is 4096 bytes Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: detected max file descriptor number: 2048 Jul 15 13:45:42.962196 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: lock engine: pthread robust mutexes Jul 15 13:45:42.963363 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: thunder lock: enabled Jul 15 13:45:42.963363 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 15 13:45:42.963363 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 15 13:45:43.002054 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: Python main interpreter initialized at 0x55f1922449c0 Jul 15 13:45:43.002054 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: python threads support enabled Jul 15 13:45:43.002054 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: your server socket listen backlog is limited to 100 connections Jul 15 13:45:43.002054 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: your mercy for graceful operations on workers is 90 seconds Jul 15 13:45:43.002563 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: mapped 403077 bytes (393 KB) for 2 cores Jul 15 13:45:43.002563 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: *** Operational MODE: preforking *** Jul 15 13:45:43.002768 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: *** uWSGI is running in multiple interpreter mode *** Jul 15 13:45:43.002768 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: spawned uWSGI master process (pid: 80118) Jul 15 13:45:43.002961 dv-cinder-850002-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 15 13:45:43.006168 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: spawned uWSGI worker 1 (pid: 80130, cores: 1) Jul 15 13:45:43.006168 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: spawned uWSGI worker 2 (pid: 80131, cores: 1) Jul 15 13:45:43.006168 dv-cinder-850002-1-iscsi devstack@keystone.service[80118]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 15 13:45:48.503106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 13:45:48.503106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Jul 15 13:45:48.503916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 13:45:48.503916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Jul 15 13:45:48.505182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 13:45:48.505182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: from cryptography.utils import int_from_bytes Jul 15 13:45:48.505740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 13:45:48.505740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: from cryptography.utils import int_from_bytes Jul 15 13:45:48.552826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 13:45:48.553048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 13:45:48.553197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] command line args: [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 13:45:48.553471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 13:45:48.553646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ================================================================================ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 13:45:48.553899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.554143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_dir = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.554330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.554511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_source = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.554688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] debug = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.554887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.555201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 13:45:48.555309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.555421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 13:45:48.555522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.555638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] command line args: [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 13:45:48.555734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.555840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 13:45:48.555943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ================================================================================ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 13:45:48.556152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] admin_token = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_config_append = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_dir = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_dir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.556826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_source = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] debug = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.557962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_config_append = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_dir = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.558932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.559931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.560076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.560236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.560429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.560625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.560789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.560927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] publish_errors = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.561999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.562153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.562308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.562469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.562622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.562775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] standard_threads = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.562939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.563090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.563243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.563395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.563395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.563395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_journal = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.563864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_json = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] publish_errors = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_stderr = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_syslog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.564983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.565116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.565274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.565409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.565540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.565672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.565803 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.565948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] standard_threads = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.566094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.566228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.566359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.566492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.566623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.566754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.566754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_journal = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.567035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.567169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_json = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.567312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.567445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_stderr = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.567445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_syslog = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.567704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.567837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 13:45:48.567969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.568945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.569920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.570893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.571963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.572951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.573890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.574952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.575898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.576888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.577020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.577158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.577416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.577555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.577765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.577899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.578867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.579973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.580248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.580453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.580602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.580763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.580914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.581062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.581226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.581373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.581518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.581664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.581806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.582015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.582679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.582836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.582992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.583144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.583304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.583442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.583608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.583767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.583920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.584076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.584242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.584452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.584612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.584777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.584938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585106 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.585994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.586153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.586299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.586463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.586618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.586768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.586926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.587968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.588956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.589919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.590887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.591962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.592101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.592235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.592368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.592628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.592767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.592767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.593897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.594988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.595920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.external = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.596969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.597112 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.597255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.password = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.597391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.597524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.token = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.597663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.598423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.598423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.598724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.598861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.598995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.external = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.599981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.600127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.600281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.600455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.password = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.600604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.600741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.token = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.600876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.601972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.602928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603204 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.603899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.604962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.605921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.606880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.607978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.608114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.608249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.608383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.608573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.608715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.608873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.609964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.610833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.611965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.612107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.612256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.612442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.612589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.612736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.612882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.613873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.614863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.615866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.616928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.617891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.618993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.619935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.620937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.621930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.622927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.623924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.624983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.625939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.626895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.627978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.628111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.628260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.628423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.628575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.628714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.628857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.629870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.630971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.631952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.632984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.633937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.634892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.635987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.636129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.636264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.636429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.636579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.636720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.636942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.637953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.638982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.639123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.639265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.639427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.639579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.639721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.639862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.640872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641294 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.641983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.642956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.643934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.644069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.644301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.644511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.645978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.646951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.647947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.648984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.649996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.650979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.651965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.652109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.652250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.652515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.652663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.652805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.652960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.653968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.654142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.654327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.654473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.654615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.654766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.654909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.655930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.656951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.657225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.658361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.658361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.658361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.658361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 13:45:48.658361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 13:45:48.671737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.672001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.672177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.673321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.673561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.673795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.681020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.682615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.710379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.710603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.711651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.711891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.856269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.856577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.906600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.906820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.906975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.907092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.975178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.975398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.975531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.975682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.975797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.975919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.976039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.976158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.976276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.976438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.976551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:48.976551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.002924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.003014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.003138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.003224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.003355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.003452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.015781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.016014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.029798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.030988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.031102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.031214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.031337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.031467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 13:45:49.048731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.048943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 13:45:49.049125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.049346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 13:45:49.049554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.049765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 13:45:49.049952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 13:45:49.050352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.050978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.051243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.061949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.062233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.062348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.062502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.062641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.062786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.062905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.063073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.071717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.072009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.075105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.075391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.075506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.075689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.075801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.075978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.076932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.077901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.078017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.078147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.105800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.107503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.107626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.107745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.109278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.109571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.113090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.113406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.113700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.114926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.115214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.115499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.121258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.121568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.123214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.123510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.126093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.126382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.126663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.126945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.127221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.127540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.127813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.128058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.128173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.128355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.128668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.129504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.129779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.130061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.130336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.130609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.148187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.148503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.150131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.150429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.153151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.155067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.156979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.157277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.158906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.159208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.163220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.164917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.166754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.168632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.170427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.170729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.171015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.171309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.171598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.171894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.172307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.172641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.172932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.173222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.173531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.173823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.191291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.192143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.193067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.193457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.193817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.194130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.194416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.194694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.202525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.203000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.204346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.204657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.204844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.205128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.208579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.208872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.209140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.209260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.209435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.209572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.209724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.209841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.210018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.210297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.223680 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.223975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.224257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.224376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.224559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.224693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.224840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.224962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.225127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.225245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.225410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.225646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.225818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.225956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.245645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.246087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.249060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.249462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.252543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.252922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.254697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.255115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.256566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.256854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.257002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.257147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.257295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.257564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.264578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.265017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.266222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.266772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.269593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.270547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.270698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.270953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.271124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.271264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.271398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.271537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.280920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.281077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.281226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.281345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.281472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.281657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.289501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.289667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 13:45:49.289790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.289914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.290944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.291954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.292952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.293077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.293195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.293325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 13:45:49.332599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f1922449c0 pid: 80131 (default app) Jul 15 13:45:49.333060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f1922449c0 pid: 80130 (default app) Jul 15 13:45:49.336259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4784b-9163-4841-a141-01c127f449f0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:45:49.336578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4784b-9163-4841-a141-01c127f449f0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:45:49.336857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4784b-9163-4841-a141-01c127f449f0 None None] PATH_INFO: `/v3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:45:49.341143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Fri Jul 15 13:45:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:45:57.053286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:45:57.053872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:45:57.053872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:45:57.058169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:45:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:45:57.063765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:45:57.064227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:45:57.064324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:45:57.133328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: INFO dbcounter [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] Registered counter for database keystone Jul 15 13:45:57.154746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG oslo_db.sqlalchemy.engines [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80131) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 13:45:57.161168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writer thread running {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 13:45:57.344961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80131) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 13:45:57.345296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80131) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 13:45:57.403218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:45:57.404937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:45:57.531473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:45:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 13:45:57.683916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:45:57.752656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: INFO dbcounter [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] Registered counter for database keystone Jul 15 13:45:57.772473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG oslo_db.sqlalchemy.engines [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80130) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 13:45:57.778596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writer thread running {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 13:45:57.898397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd1ad0-f753-426c-8c14-50ed706a4ba5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:45:57.902773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:45:57.903458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:45:57.903741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:45:57.904020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:45:57.913466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:45:57.965857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 13:45:57.966120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747d8bd-f6c9-46d5-8cb4-211bcf6cac8a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:45:57.977606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Fri Jul 15 13:45:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 13:45:57.983686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:45:57.998429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bf2df-8c91-4ea4-9e75-d832f828f01a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:45:58.002895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:45:58.004588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:45:58.004874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:45:58.005150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] PATH_INFO: `/v3/projects/8618b94bef7144e799c6d51bb1ef841f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:45:58.006657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] RBAC: Authorizing `identity:get_project(project_id=8618b94bef7144e799c6d51bb1ef841f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:45:58.060167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 13:45:58.060526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e53e2-ba07-4f70-972c-4ceeba545dac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:45:58.065182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Fri Jul 15 13:45:57 2022] GET /identity/v3/projects/8618b94bef7144e799c6d51bb1ef841f => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:00.068291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69011c93-5f58-4fb0-a8a7-e08ab2e63034 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:00.068832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69011c93-5f58-4fb0-a8a7-e08ab2e63034 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:00.068933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69011c93-5f58-4fb0-a8a7-e08ab2e63034 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:00.072998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:00.078453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:00.078865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:00.079019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:00.098862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:00.100380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:00.108195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 13:46:00.129921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69011c93-5f58-4fb0-a8a7-e08ab2e63034 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:00.142138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69011c93-5f58-4fb0-a8a7-e08ab2e63034 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:00.146918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5654da-456f-4982-85fd-3515103e39a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:00.147573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5654da-456f-4982-85fd-3515103e39a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:00.147863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5654da-456f-4982-85fd-3515103e39a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:00.148190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5654da-456f-4982-85fd-3515103e39a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:00.156340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5654da-456f-4982-85fd-3515103e39a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:00.158149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5654da-456f-4982-85fd-3515103e39a9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:00.167805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Fri Jul 15 13:46:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 13:46:00.173544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:00.182693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ec906-c61b-4127-88c1-3161ffe5a678 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:00.187050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca84db-0f3b-4e7b-9965-43c2f15ec4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:00.187919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca84db-0f3b-4e7b-9965-43c2f15ec4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:00.188206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca84db-0f3b-4e7b-9965-43c2f15ec4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:00.188519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca84db-0f3b-4e7b-9965-43c2f15ec4b4 admin admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:00.189917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca84db-0f3b-4e7b-9965-43c2f15ec4b4 admin admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:00.192322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aca84db-0f3b-4e7b-9965-43c2f15ec4b4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:00.194081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Fri Jul 15 13:46:00 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:02.302071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db27715-3ef0-4932-b1cd-879b1df3b8ee None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.302610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db27715-3ef0-4932-b1cd-879b1df3b8ee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.302706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db27715-3ef0-4932-b1cd-879b1df3b8ee None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.304070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:02.304599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a984a1-020c-499e-b4d2-09d4bdd6e7a7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.304985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a984a1-020c-499e-b4d2-09d4bdd6e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.305835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a984a1-020c-499e-b4d2-09d4bdd6e7a7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.306612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:02.310846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.310846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.310846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.312061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.312360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.312676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.329450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:02.330652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:02.352046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG passlib.handlers.bcrypt [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80130) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 13:46:02.352505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG passlib.handlers.bcrypt [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80130) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 13:46:02.397866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:02.399448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:02.412741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 7/12] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:02.435635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:02.460512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:02.504709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 6/13] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:02.527358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:02.543728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6597c-1502-48fc-8baf-7fc894136e16 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:02.578167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:02.578964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.579250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.579526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.592062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:02.603617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:02.604713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:02.605690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 8/14] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:02.610883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:02.632317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f6194-e97d-46f1-a12c-5ab942fc4d11 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:02.642053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:02.674356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:02.674727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.674998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.675409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.699500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:02.719213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c12361-b634-4da0-9638-fe0408089b01 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:02.751232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:02.753466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:02.756245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:02.756950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.757235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.757511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.758273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:02.758580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:02.759967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:02.772439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 9/15] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:02.778134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:02.867892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 7/16] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 13:46:02.873729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:02.877576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:02.881562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd939a9b-083c-44da-87ee-98c85ff1bb1a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:02.915923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:02.917107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:02.917392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:02.917674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:02.918903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:02.920413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:02.922284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 10/17] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:02 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:02.928734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:02.944416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:02.977302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d5700-e1bc-4be6-9f3b-248f7b39c307 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:03.013357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:03.014157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:03.014435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:03.014729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:03.033320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:03.033524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d926-13aa-47b1-9803-8ff978ed2cba None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:03.034797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:46:03.057957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:03.058990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:03.059990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:46:03.059990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:46:03.059990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:46:03.059990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:03.059990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 8/18] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:03.065111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:03.070998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:03.071802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:03.072091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:03.072374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:03.073612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:03.075079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:03.076775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 11/19] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:03.082585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:03.129587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:03.172828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:03.179208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfad50-69b0-4f55-aa7f-e18178cbc567 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:03.193977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dba8da-bd53-4bee-89e5-b6a3cc807bc3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:03.215423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:03.216128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:03.216437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:03.216722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:03.217468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:03.217776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:03.219208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c3ae7-10bc-4f73-bfea-9ebaabebc154 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:03.231126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:03.231830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:03.232096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:03.232373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:03.233259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.domain.id=default, user.id=0802dbaa2668431bbb6defab89b49e81) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:03.233571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:03.235033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e7c97-7134-4371-a07c-f94dd3d0300e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:03.239664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 9/20] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:03.243451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 12/21] 10.222.0.16 () {56 vars in 1322 bytes} [Fri Jul 15 13:46:03 2022] GET /identity/v3/role_assignments?user.id=0802dbaa2668431bbb6defab89b49e81&scope.domain.id=default&role.id=10895c39249448e4ac3616747cafa76b => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:05.661242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dec186-49ae-4968-95d8-953f558a1c2f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:05.662249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dec186-49ae-4968-95d8-953f558a1c2f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:05.662837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dec186-49ae-4968-95d8-953f558a1c2f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:05.664787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 10/22] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:05.670114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:05.671010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:05.671484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:05.683717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae74dd-c3ea-42a0-83c1-323291b2af44 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:05.684637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae74dd-c3ea-42a0-83c1-323291b2af44 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:05.685395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae74dd-c3ea-42a0-83c1-323291b2af44 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:05.694464 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 11/23] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:05.710205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:05.711184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:05.711963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:05.731702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:05.739642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:05.761491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:05.763174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:05.911063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 13/24] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:05.926762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 12/25] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:05.953884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:05.994948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:06.070712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:06.127636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:06.133137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621628cf-ec0b-4bc3-89c6-07ab06efc348 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:06.158864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1335a-eead-4aca-bf81-1cca54904fa7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:06.184045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:06.185025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.185382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.185878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.198478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:06.200358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:06.202236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:06.203242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.203681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.204175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.217997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f16caa7c-2b6f-4f39-8ff5-ff9a8aa30902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:46:06.220014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:06.220995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:06.221767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:46:06.221767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:46:06.221767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:46:06.221767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:06.221767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 13/26] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:06.223576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092edebf-039f-4d27-b1d2-692108dd5e83 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.223860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092edebf-039f-4d27-b1d2-692108dd5e83 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.224137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092edebf-039f-4d27-b1d2-692108dd5e83 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.225508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 14/27] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:06.226711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552e817-fbac-46e9-ac60-5104873d1aaf None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.227047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552e817-fbac-46e9-ac60-5104873d1aaf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.227318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552e817-fbac-46e9-ac60-5104873d1aaf None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.228590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 15/28] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:06.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:06.232125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:06.233076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:06.235062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4c034-7971-4859-9291-8173b848c9a4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.235314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 14/29] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:06.236284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:06.236954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4c034-7971-4859-9291-8173b848c9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.237316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4c034-7971-4859-9291-8173b848c9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.256510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b3d4c034-7971-4859-9291-8173b848c9a4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:06.257861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b3d4c034-7971-4859-9291-8173b848c9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:06.347641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 16/30] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:06.350282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.350680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.351060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.352195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301cb2e-f22c-452b-a76b-2fe58ab5ebc9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:06.373384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:06.375100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:06.395059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:06.396018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.396471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.396911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.397816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:06.398321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:06.399951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:06.402208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:06.407115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:06.423740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 15/31] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:06.425041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:06.469244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 17/32] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:06.477703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:06.545901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45544d6-c3b3-4fa8-a55b-3881c1860882 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:06.593322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:06.594600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.595094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.595690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.597191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:06.597857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:06.600240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:06.605809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d34d4-e438-481e-b84e-f0d3d6785336 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:06.615830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 16/33] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:06.617103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:06.661553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:06.662891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.664035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.664212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.665986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:06.669237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:06.723614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:06.810889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549d68-6509-46c3-8ea0-03199d3502b5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:06.868620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 18/34] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jul 15 13:46:06 2022] POST /identity/v3/roles => generated 225 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:06.869898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:06.887104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b02ea0-943f-4242-a22d-0c6ef99daf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:06.888677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b02ea0-943f-4242-a22d-0c6ef99daf9c None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:06.889403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b02ea0-943f-4242-a22d-0c6ef99daf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:06.890071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b02ea0-943f-4242-a22d-0c6ef99daf9c None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:06.891571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b02ea0-943f-4242-a22d-0c6ef99daf9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:06.894507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b02ea0-943f-4242-a22d-0c6ef99daf9c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:06.903241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:07.183639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b6c3a-a9d6-4afa-87dc-bf4bdcf3260a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:07.269001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:07.270722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.271186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.271642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.272934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:07.275589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:07.305531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 19/35] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jul 15 13:46:06 2022] POST /identity/v3/roles => generated 219 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:07.306904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:07.343430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 17/36] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:46:06 2022] POST /identity/v3/projects => generated 301 bytes in 726 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:07.458437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:07.483477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab119b-d4bc-4a09-965d-67a30b32f427 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.483778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab119b-d4bc-4a09-965d-67a30b32f427 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.484049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab119b-d4bc-4a09-965d-67a30b32f427 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.485621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 18/37] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:07.489654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ae192-4723-48f5-97ae-c970c1204d75 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:07.492981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2c89-9e6f-4c99-a35d-9676c71c57b5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.493264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2c89-9e6f-4c99-a35d-9676c71c57b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.493553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de2c89-9e6f-4c99-a35d-9676c71c57b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.544433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-62de2c89-9e6f-4c99-a35d-9676c71c57b5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:07.544433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-62de2c89-9e6f-4c99-a35d-9676c71c57b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:07.562886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:07.564538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.564834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.565141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.568379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:07.573818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:07.577039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 20/38] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:07 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:07.584693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:07.661629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 19/39] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:07.662783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e6227-9b9f-4698-b7c2-1daa6f382619 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.663078 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e6227-9b9f-4698-b7c2-1daa6f382619 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.663363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e6227-9b9f-4698-b7c2-1daa6f382619 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.664743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 20/40] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:07.670442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.670743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.670947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.689914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:07.691234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:07.724190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:07.725064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f571708-4a9d-41c6-98ca-f8a7d25adc27 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:07.762845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:07.763873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.764355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.764765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.766401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:07.768042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:07.769041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 21/41] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:07.770266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 21/42] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:07.770444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:07.777540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:07.795515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:07.892133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea472c-6b79-4063-a7dd-8eb57f41ceb2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:07.897708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58b1b0-79ee-4ee2-8a57-48954eaf381a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:07.930258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:07.930952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.931242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.931527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.932301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:07.933912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:07.934747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:07.935656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:07.935943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:07.936231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] PATH_INFO: `/v3/domains/default/users/0802dbaa2668431bbb6defab89b49e81/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:07.938394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0802dbaa2668431bbb6defab89b49e81, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:07.939990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:07.950008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 22/43] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jul 15 13:46:07 2022] POST /identity/v3/roles => generated 223 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:07.950937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:07.966068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 22/44] 10.222.0.16 () {58 vars in 1361 bytes} [Fri Jul 15 13:46:07 2022] PUT /identity/v3/domains/default/users/0802dbaa2668431bbb6defab89b49e81/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 13:46:07.998671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:08.094852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95660518-72b2-4dc0-8222-b3a015d6c76c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:08.139912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ca69e-1c4e-4dc9-a273-a3492ef45857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:08.140761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ca69e-1c4e-4dc9-a273-a3492ef45857 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:08.141051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ca69e-1c4e-4dc9-a273-a3492ef45857 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:08.141333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ca69e-1c4e-4dc9-a273-a3492ef45857 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:08.145870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ca69e-1c4e-4dc9-a273-a3492ef45857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:08.145870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ca69e-1c4e-4dc9-a273-a3492ef45857 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:08.145870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 23/45] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:08.152573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:08.247667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:08.274374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea3f01-8462-4b56-a849-2d9a341b1370 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:08.308450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f5066-9498-4d4f-a037-bca1ee2b960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:08.309127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5066-9498-4d4f-a037-bca1ee2b960c None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:08.309420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5066-9498-4d4f-a037-bca1ee2b960c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:08.309708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f5066-9498-4d4f-a037-bca1ee2b960c None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:08.310435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5066-9498-4d4f-a037-bca1ee2b960c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:08.311917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f5066-9498-4d4f-a037-bca1ee2b960c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:08.340850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:46:08 2022] POST /identity/v3/projects => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:10.015907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4172bfc-5280-447f-9b2f-fdb1df01fb3a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.016454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4172bfc-5280-447f-9b2f-fdb1df01fb3a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.016560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4172bfc-5280-447f-9b2f-fdb1df01fb3a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.017888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:10.023409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.023834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.023977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.041335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:10.042842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:10.112236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:10.135046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4172bfc-5280-447f-9b2f-fdb1df01fb3a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.246263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:10.255127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4172bfc-5280-447f-9b2f-fdb1df01fb3a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:10.291069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad989d5-4734-4c5f-962f-f0bac765954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:10.291795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad989d5-4734-4c5f-962f-f0bac765954b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.292089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad989d5-4734-4c5f-962f-f0bac765954b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.292375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad989d5-4734-4c5f-962f-f0bac765954b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.303821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad989d5-4734-4c5f-962f-f0bac765954b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8ad989d5-4734-4c5f-962f-f0bac765954b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:10.311698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:10.312576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:10.313301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:10.319073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.376936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:10.392147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8f1f0-4127-4793-819d-11c9ff9ee5ae None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.393341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8f1f0-4127-4793-819d-11c9ff9ee5ae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.393630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8f1f0-4127-4793-819d-11c9ff9ee5ae None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.395558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 26/50] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:10.400964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.401254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.401540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.418597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:10.419812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:10.425734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05a7c-8c7b-436a-a3ac-8c63d2a1ae40 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:10.460335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:10.461055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.461347 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.461629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.462391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:10.462700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:10.464129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:10.477179 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 25/51] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:10.483176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.488985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 27/52] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:10.511417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.590792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430e1e-5c3f-445e-bf6f-ee3ee798dcec None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:10.599969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:10.622785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addd590-8cfb-47ea-8087-18a4c5b1b451 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:10.661984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:10.662796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.663083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.663362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.664664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:10.666170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:10.667903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 28/53] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:10.673898 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.719085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:10.737945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:10.739377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.739656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.739941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.741204 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:10.742681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:10.744344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 26/54] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:10 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:10.749969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.778385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ebbe0-87be-409f-824d-0e8d657844ab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:10.814320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:10.814990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.815390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.815561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.816323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:10.817840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:10.843743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 29/55] 10.222.0.16 () {60 vars in 1140 bytes} [Fri Jul 15 13:46:10 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:46:10.859824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc7c96-f1bd-4567-b0f5-2bf5a40dc741 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:10.899611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a71f4-108a-4b9f-85ad-b4bc7b164f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:10.900487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a71f4-108a-4b9f-85ad-b4bc7b164f15 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:10.900784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a71f4-108a-4b9f-85ad-b4bc7b164f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:10.901067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a71f4-108a-4b9f-85ad-b4bc7b164f15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:10.902441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a71f4-108a-4b9f-85ad-b4bc7b164f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:10.903949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a71f4-108a-4b9f-85ad-b4bc7b164f15 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:10.905741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 27/56] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:10.912432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:10.922488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:11.027641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334413c2-085c-40d4-b1c9-18721277a555 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:11.066394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:11.067064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:11.067354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:11.067639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:11.068511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.domain.id=default, user.id=0802dbaa2668431bbb6defab89b49e81) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:11.068831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:11.070299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378992-a2fb-48c8-a737-250fcb003509 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:11.076331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 30/57] 10.222.0.16 () {56 vars in 1322 bytes} [Fri Jul 15 13:46:10 2022] GET /identity/v3/role_assignments?user.id=0802dbaa2668431bbb6defab89b49e81&scope.domain.id=default&role.id=10895c39249448e4ac3616747cafa76b => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:12.884722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ebb4-0885-4535-b79f-33d06f8d209c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:12.885259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ebb4-0885-4535-b79f-33d06f8d209c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:12.885368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551ebb4-0885-4535-b79f-33d06f8d209c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:12.886725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 28/58] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:12.892922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:12.893473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:12.893473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:12.913297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:12.914549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:12.980712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:12.985484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 31/59] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:13.009289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ebb4-0885-4535-b79f-33d06f8d209c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:13.076770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:13.122444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551ebb4-0885-4535-b79f-33d06f8d209c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:13.155781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bfc97-92cc-4c1e-b974-d1b7399a6fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:13.156548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bfc97-92cc-4c1e-b974-d1b7399a6fef None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:13.156833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bfc97-92cc-4c1e-b974-d1b7399a6fef None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:13.157111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bfc97-92cc-4c1e-b974-d1b7399a6fef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:13.158385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bfc97-92cc-4c1e-b974-d1b7399a6fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:13.159913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bfc97-92cc-4c1e-b974-d1b7399a6fef None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:13.161699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 29/60] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:13.167350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:13.264236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217fd28-6140-44a6-8f64-6532ed50beca None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:13.297282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048d1a6-2a65-417e-8ade-e1df3ac1286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:13.297946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048d1a6-2a65-417e-8ade-e1df3ac1286b None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:13.298231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048d1a6-2a65-417e-8ade-e1df3ac1286b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:13.298509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048d1a6-2a65-417e-8ade-e1df3ac1286b None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:13.299226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048d1a6-2a65-417e-8ade-e1df3ac1286b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:13.301247 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048d1a6-2a65-417e-8ade-e1df3ac1286b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:13.328932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 32/61] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:13 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:16.509961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e00bb3-6b30-42f9-a1c3-28b3a6796a5c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:16.510491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e00bb3-6b30-42f9-a1c3-28b3a6796a5c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:16.510491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e00bb3-6b30-42f9-a1c3-28b3a6796a5c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:16.512039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 30/62] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:16.526047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7b83-6c57-4510-b09e-2c566afa30fd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:16.526588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7b83-6c57-4510-b09e-2c566afa30fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:16.526588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf7b83-6c57-4510-b09e-2c566afa30fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:16.553158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-76cf7b83-6c57-4510-b09e-2c566afa30fd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:16.554513 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76cf7b83-6c57-4510-b09e-2c566afa30fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:16.632765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:16.660116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 33/63] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:16.697282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e00bb3-6b30-42f9-a1c3-28b3a6796a5c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:16.869453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e00bb3-6b30-42f9-a1c3-28b3a6796a5c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:16.899621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0e9f-00db-475a-9688-8fb264fd4eaa None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:16.900253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0e9f-00db-475a-9688-8fb264fd4eaa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:16.900779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b0e9f-00db-475a-9688-8fb264fd4eaa None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:16.902665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 34/64] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:16.908947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84073-4e51-4c4c-81e1-f337967008c7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:16.909616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84073-4e51-4c4c-81e1-f337967008c7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:16.910656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a84073-4e51-4c4c-81e1-f337967008c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:16.920772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:16.927316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96037b30-9065-4862-a384-e35248b72d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:16.928513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96037b30-9065-4862-a384-e35248b72d29 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:16.929082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96037b30-9065-4862-a384-e35248b72d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:16.929385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96037b30-9065-4862-a384-e35248b72d29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:16.930397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f3a84073-4e51-4c4c-81e1-f337967008c7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:16.931851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3a84073-4e51-4c4c-81e1-f337967008c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:16.947004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96037b30-9065-4862-a384-e35248b72d29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-96037b30-9065-4862-a384-e35248b72d29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:16.964451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:16.965496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:16.966223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 31/65] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:16.967233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f9ebf-0bfa-4712-a5e5-51ea70e43994 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:16.967764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f9ebf-0bfa-4712-a5e5-51ea70e43994 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:16.968334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f9ebf-0bfa-4712-a5e5-51ea70e43994 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:16.970953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 32/66] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:16.974049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f9ebf-0bfa-4712-a5e5-51ea70e43994 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.028260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 35/67] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:17.029956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.030530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.031234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.059838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:17.061423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:17.150982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f9ebf-0bfa-4712-a5e5-51ea70e43994 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.168416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 36/68] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:17.168416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.194911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.195609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.195903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.196189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.197000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:17.197328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:17.198726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53774bf5-7d6e-46a3-9d66-9e5cd5f3f40c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:17.209206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 33/69] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:17.210562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4d1f-6b99-4fd9-adce-7f1555b78ee9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.210856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4d1f-6b99-4fd9-adce-7f1555b78ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.211142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4d1f-6b99-4fd9-adce-7f1555b78ee9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.212462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 34/70] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:17.213431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea4d1f-6b99-4fd9-adce-7f1555b78ee9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.250677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:17.282300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:17.318743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c506af-62c4-4e8b-bd82-e0d00a171f39 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.357601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea4d1f-6b99-4fd9-adce-7f1555b78ee9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.363424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef5bf-69e9-43e7-a325-b75e1bcb13e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.364482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef5bf-69e9-43e7-a325-b75e1bcb13e5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.364780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef5bf-69e9-43e7-a325-b75e1bcb13e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.365123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef5bf-69e9-43e7-a325-b75e1bcb13e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.380057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef5bf-69e9-43e7-a325-b75e1bcb13e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-101ef5bf-69e9-43e7-a325-b75e1bcb13e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:17.388415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:17.389318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:17.390042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 37/71] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:17.391748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.391748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.391748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.409078 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:17.410229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:17.414536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.415432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.415848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.416346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.431839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:17.442476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:17.444171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:17.445729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 35/72] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:17.448218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.504980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 38/73] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:17.505862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.547627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:17.592938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efbbde-0136-452f-ae0f-8480cad68caf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.596583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:17.645318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445cbfd-d4d7-4af6-a6e6-fd662d7607f8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.649024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.649928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.650219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.650502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.668734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:17.670177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:17.672000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 36/74] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:17.672851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.694993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.695694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.695986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.696266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.697104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:17.697425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:17.698887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:17.708333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 39/75] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:17.709254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.782720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f698bd6-ac73-40f4-bcd7-eb0d9a3d2d68 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.818784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.819715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.820017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.820302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.821126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:17.821438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:17.822837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a9f3a-411a-495b-b7cc-147b162876a7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:17.823245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:17.827810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a963c-cf4b-435a-987d-7851d3957935 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:17.831574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 37/76] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:17.833608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888a718-00c9-4b43-b184-ec733386133c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.833907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888a718-00c9-4b43-b184-ec733386133c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.834195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888a718-00c9-4b43-b184-ec733386133c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.835656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 38/77] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:17.840680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888a718-00c9-4b43-b184-ec733386133c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.868072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:17.868305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:17.869204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:17.869526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:17.869845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:17.880872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:17.888807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:17.889632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:17.890359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 40/78] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:17.890955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:17.973034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888a718-00c9-4b43-b184-ec733386133c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.007623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead6977-dae0-47d3-ad49-97ed87bfde5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.008296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead6977-dae0-47d3-ad49-97ed87bfde5a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.008629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead6977-dae0-47d3-ad49-97ed87bfde5a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.008917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead6977-dae0-47d3-ad49-97ed87bfde5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.009777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193060f0-5080-447f-9e66-83d52ede8f8d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.051824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.052761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.053058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.053488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.054897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.056413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.062904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 41/79] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:18.062904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.088052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead6977-dae0-47d3-ad49-97ed87bfde5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.089593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead6977-dae0-47d3-ad49-97ed87bfde5a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.091440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.119617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.169291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fe9c5-2341-4720-9c75-fea3fa37107d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.205942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 39/80] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:18.207409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.207704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.208032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.212165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.213292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.213840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.214130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.215396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.216932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.218630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 42/81] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:18.219412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.232695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:18.233925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:18.309523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 40/82] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:18.310425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.328828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.336311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af5116-c95c-4eac-8f81-dea524f8a92b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.370002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.373591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.374262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.374547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.374825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.375586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:18.375890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.377313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.385627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 43/83] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:18.386422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.430950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21413b0d-352d-4891-b3bb-1485e630df96 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.466534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.467202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.467489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.467773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.492305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adedd3-2b84-404e-ae2d-23cc2562c51d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.527358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.528233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.528560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.528846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.530044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.531519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.533285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 44/84] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:18.534055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.542577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.544084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.564437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.607078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.644690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51d392-e891-4733-80e3-548e919fb7e3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.659183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 41/85] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:18.660691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.686593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.687366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.687651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.687967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.762033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.763412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.792541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.794322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07b276-3ed2-4d4c-a5fa-42f9e8fb5308 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.834330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.835152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.835436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.835713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.837030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:18.838538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:18.840286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 42/86] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:18.841138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.872666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:18.876919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 45/87] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:18.877877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:18.959515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f7216-5217-4bb5-870f-67786d9474e5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.990505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b84ec8-919b-4a78-b5fc-af8c864b768d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:18.997182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:18.998073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:18.998359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:18.998673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:18.999915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.001415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.003111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 43/88] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:19.004465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.017165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:19.029929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.030617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.030907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.031193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.032069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=1c3a6fffeecf41fb83087b2815dfe4b8) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:19.032420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.033970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.040641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 46/89] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:18 2022] GET /identity/v3/role_assignments?user.id=1c3a6fffeecf41fb83087b2815dfe4b8&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:19.041507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.112659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:19.126698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07719049-46ad-4b1e-9b0e-f9f237351000 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.177606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.178774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.179066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.179350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.180761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.182289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.183630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab03e1-17d7-427e-883e-5cff70d47425 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.184128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 44/90] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:19.184984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.235372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.238115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.238115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.238115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.238115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.239315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.264337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 47/91] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:19 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:19.265227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.341562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ba9b8-1e01-4767-b77e-b0f9db8d3387 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.368922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:19.387693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.389048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.389689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.390417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.392516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.394416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.396877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 45/92] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:19.397934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.437342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:19.504314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c665dee-ef1d-4a97-a4d4-15a1ea900423 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.534667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752eda6-5ac7-469d-9785-c5952706d374 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.554645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.556005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.556005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.556005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.589518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.590200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.590487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.590772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.591673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] RBAC: Adding query filter params (role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=1c3a6fffeecf41fb83087b2815dfe4b8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:19.591935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.593423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.603121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 46/93] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/role_assignments?user.id=1c3a6fffeecf41fb83087b2815dfe4b8&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=c41b65addaec48bfb92f135a2a724132 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:19.604104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.669246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.670782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.731195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:19.756598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:19.770187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54e8a1-a6f2-409e-a22d-37819546604f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.814052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.814728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.815018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.815303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.815853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 48/94] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:19.816369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=0802dbaa2668431bbb6defab89b49e81) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:19.816734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.818291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.821451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:19.824784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 47/95] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/role_assignments?user.id=0802dbaa2668431bbb6defab89b49e81&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:19.939911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f764027-0295-4f44-bee5-f211b1e046ae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:19.979392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f8e3a-2760-4f56-8481-c7b844fc3adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:19.980284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f8e3a-2760-4f56-8481-c7b844fc3adc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:19.980607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f8e3a-2760-4f56-8481-c7b844fc3adc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:19.980893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f8e3a-2760-4f56-8481-c7b844fc3adc None admin] PATH_INFO: `/v3/projects/19e74baba5a9416a98b86a80b07f4140` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:19.982121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f8e3a-2760-4f56-8481-c7b844fc3adc None admin] RBAC: Authorizing `identity:get_project(project_id=19e74baba5a9416a98b86a80b07f4140)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:19.983600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f8e3a-2760-4f56-8481-c7b844fc3adc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:19.985560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 49/96] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/projects/19e74baba5a9416a98b86a80b07f4140 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:19.991978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:20.138489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8a38-0a06-4972-a9f7-bd802d786d5e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:20.202098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:20.202695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:20.202988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:20.203275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:20.204130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=19e74baba5a9416a98b86a80b07f4140, user.id=1c3a6fffeecf41fb83087b2815dfe4b8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:20.204472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:20.205946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9d539-1cb7-4659-b352-0b404f151fc1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:20.214896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 48/97] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:19 2022] GET /identity/v3/role_assignments?user.id=1c3a6fffeecf41fb83087b2815dfe4b8&scope.project.id=19e74baba5a9416a98b86a80b07f4140&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:21.538134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f84f21-824d-49d8-bea8-f44d3e161630 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:21.538636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f84f21-824d-49d8-bea8-f44d3e161630 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:21.538636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f84f21-824d-49d8-bea8-f44d3e161630 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:21.539993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 50/98] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:21.545617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:21.546071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:21.546263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:21.563865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:21.565133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:21.587433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:21.666749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 49/99] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:21.699773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f84f21-824d-49d8-bea8-f44d3e161630 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:21.718502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:21.843282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f84f21-824d-49d8-bea8-f44d3e161630 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:21.898139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:21.898958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:21.899248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:21.899531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:21.917870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:21.925834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:21.926710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:21.927483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 51/100] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:21.933272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:22.072281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f2059-c7cf-40cb-94a5-ee6e72d2c0c1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:22.120358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:22.121456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:22.121952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:22.122460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:22.123448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:22.123961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:22.125691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:22.144978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 50/101] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:22.155393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:22.331349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:22.399408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4dd07b-ba89-4c50-979d-c404537c4540 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:22.453250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:22.454130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:22.454410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:22.454682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:22.461185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:22.462660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:22.464567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 52/102] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:22 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:22.473039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:22.490798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:22.604238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af352b0-f58e-4b95-a87f-48d939f09972 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:22.649380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b990a-85c3-4f26-a2ec-a9f5d11ca6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:22.650667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b990a-85c3-4f26-a2ec-a9f5d11ca6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:22.651182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b990a-85c3-4f26-a2ec-a9f5d11ca6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:22.651662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b990a-85c3-4f26-a2ec-a9f5d11ca6e8 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:22.653125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b990a-85c3-4f26-a2ec-a9f5d11ca6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:22.654891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b990a-85c3-4f26-a2ec-a9f5d11ca6e8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:22.656886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 51/103] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:22 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:22.665458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:22.803982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7cea-932f-4908-8173-de87beea7133 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:22.804766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7cea-932f-4908-8173-de87beea7133 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:22.805287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7cea-932f-4908-8173-de87beea7133 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:22.806949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 52/104] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:22.816442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da6434-96dc-464a-9b93-0b230c12c840 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:22.818423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a580-2528-4ae3-88b4-3b6817ba9e4c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:22.818937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a580-2528-4ae3-88b4-3b6817ba9e4c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:22.819546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a580-2528-4ae3-88b4-3b6817ba9e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:22.848418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:22.848418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:22.848931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:22.849163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276/users/1c3a6fffeecf41fb83087b2815dfe4b8/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:22.851357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e19f2b19b0472c87c1688310368276, user_id=1c3a6fffeecf41fb83087b2815dfe4b8, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:22.852948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:22.869621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9d08a580-2528-4ae3-88b4-3b6817ba9e4c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:22.871350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d08a580-2528-4ae3-88b4-3b6817ba9e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:22.877034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:22.883869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 53/105] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:22 2022] PUT /identity/v3/projects/f7e19f2b19b0472c87c1688310368276/users/1c3a6fffeecf41fb83087b2815dfe4b8/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:22.982510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 53/106] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:23.040665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:23.063075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13f9bc-9f6c-4bf9-b7da-72c05a1c111d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.063773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13f9bc-9f6c-4bf9-b7da-72c05a1c111d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.064433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13f9bc-9f6c-4bf9-b7da-72c05a1c111d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.066209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 54/107] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:23.078427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.079205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.079813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.120416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:23.120416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:23.121951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:23.260038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de61b0-1b2c-4365-acf3-b02a7dd5a542 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:23.293470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 55/108] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:23.319187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:23.320037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.320360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.320669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.321985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:23.323657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:23.325548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 54/109] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:23.332124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:23.342390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:23.500434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:23.534477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176ec5-dd30-4a4b-87c5-c9f785e36e8e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:23.553067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ee12-9c99-47ad-8479-d63cd525f7f3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:23.582998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca60a72-87c9-4f7e-8eb9-44cc033d040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:23.584678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca60a72-87c9-4f7e-8eb9-44cc033d040a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.584879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca60a72-87c9-4f7e-8eb9-44cc033d040a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.585485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca60a72-87c9-4f7e-8eb9-44cc033d040a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.592446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f7ba2-0ecf-4746-998c-d9ce374e69b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:23.593233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7ba2-0ecf-4746-998c-d9ce374e69b7 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.593597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7ba2-0ecf-4746-998c-d9ce374e69b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.593952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f7ba2-0ecf-4746-998c-d9ce374e69b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.594768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f7ba2-0ecf-4746-998c-d9ce374e69b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:23.596383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f7ba2-0ecf-4746-998c-d9ce374e69b7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:23.601990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca60a72-87c9-4f7e-8eb9-44cc033d040a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:23.606615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0ca60a72-87c9-4f7e-8eb9-44cc033d040a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:23.615216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:23.616192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:23.617511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 55/110] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:23.618549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d53c-938a-43a2-9b90-c6c5e89def64 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.618829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d53c-938a-43a2-9b90-c6c5e89def64 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.618980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d53c-938a-43a2-9b90-c6c5e89def64 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.623892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 56/111] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:23.623892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01637e7e-494f-4a34-9d70-0644e5a83052 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.623892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01637e7e-494f-4a34-9d70-0644e5a83052 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.623892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01637e7e-494f-4a34-9d70-0644e5a83052 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.623892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 57/112] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:23.624454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01637e7e-494f-4a34-9d70-0644e5a83052 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:23.636094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 56/113] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:46:23 2022] POST /identity/v3/projects => generated 302 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:23.638006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdc2f-842e-4563-b331-713a5f4f8a6d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.638535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdc2f-842e-4563-b331-713a5f4f8a6d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.639075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdc2f-842e-4563-b331-713a5f4f8a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.663499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f11cdc2f-842e-4563-b331-713a5f4f8a6d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:23.665333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f11cdc2f-842e-4563-b331-713a5f4f8a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:23.763657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 57/114] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:23.771003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceeeda7-ff8e-4c38-8a73-da10ac19fcbd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.771592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceeeda7-ff8e-4c38-8a73-da10ac19fcbd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.772074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceeeda7-ff8e-4c38-8a73-da10ac19fcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.777707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01637e7e-494f-4a34-9d70-0644e5a83052 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:23.820269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cceeeda7-ff8e-4c38-8a73-da10ac19fcbd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:23.822087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cceeeda7-ff8e-4c38-8a73-da10ac19fcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:23.843288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:23.855128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:23.855893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.856297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.856497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.857358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:23.857660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:23.859568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:23.878406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 58/115] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:23.881525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:23.971859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 58/116] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:23.973946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba46592-b7b3-43fd-8bb8-157d7a5aa38b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:23.974239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba46592-b7b3-43fd-8bb8-157d7a5aa38b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:23.974570 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba46592-b7b3-43fd-8bb8-157d7a5aa38b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:23.975907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 59/117] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:23.976682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba46592-b7b3-43fd-8bb8-157d7a5aa38b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.036834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.038652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c3d7b-1aef-4525-a38c-3fb4c80802c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.089307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d720b-8b41-4770-a181-2bba02b99edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.090079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d720b-8b41-4770-a181-2bba02b99edd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.090361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d720b-8b41-4770-a181-2bba02b99edd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.090635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d720b-8b41-4770-a181-2bba02b99edd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.099126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba46592-b7b3-43fd-8bb8-157d7a5aa38b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.110012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d720b-8b41-4770-a181-2bba02b99edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-231d720b-8b41-4770-a181-2bba02b99edd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:24.122643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:24.123547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:24.124278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:24.126159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 59/118] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:24.127529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.127821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.128104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.153750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:24.153965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.154060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:24.154147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.154147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.154147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.156959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:24.160080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:24.162442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 60/119] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:23 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:24.163555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.221905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.249831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 60/120] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:24.257575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.335703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae5442-e862-4015-bad3-75eb2ef057aa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.356422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.381297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.382353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.382651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.382932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.398802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:24.403839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56970376-f5b5-4afd-bae7-7c953c06e7ef None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:24.409146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:24.410614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 61/121] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:24.417000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.450451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.451517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.452081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.452684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.453729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:24.454273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:24.455910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:24.465586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 61/122] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:24.470597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.529005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.553688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233d098-125d-468a-9d04-3c7457f11fca None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.592008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.592939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.593231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.593516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.595301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:24.598000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:24.600849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 62/123] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:24.601753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.615976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76ddf-eecc-4e2d-bf61-243ef04185ee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.664128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.667197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.668788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.669284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.669781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.687671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:24.698397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:24.699503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:24.700455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:24.703600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 62/124] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:24.705204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.728890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc4374-b15f-425d-bcc8-3502c5454995 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.765258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.765952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.766337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.766715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.767573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:24.767982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:24.770403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:24.780369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 63/125] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:24.782008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.844896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34b69e-6daa-40fb-b7d9-d0cf9559ca8d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.848144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.903600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.905456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.905985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.906508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] PATH_INFO: `/v3/users/108478f2d096419c908833ecfac69803` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.929433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] RBAC: Authorizing `identity:get_user(user_id=108478f2d096419c908833ecfac69803)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:24.931578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:24.934237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 63/126] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/users/108478f2d096419c908833ecfac69803 => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:24.935045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dff17a-444e-4543-8d1b-33ca1a0f72ad None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:24.935522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:24.946515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:24.979440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:24.980361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:24.980669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:24.980948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276/users/1c3a6fffeecf41fb83087b2815dfe4b8/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:24.985631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e19f2b19b0472c87c1688310368276, user_id=1c3a6fffeecf41fb83087b2815dfe4b8, role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:24.987158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.005633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 64/127] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:24 2022] PUT /identity/v3/projects/f7e19f2b19b0472c87c1688310368276/users/1c3a6fffeecf41fb83087b2815dfe4b8/roles/c41b65addaec48bfb92f135a2a724132 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:25.006677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.070529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b2c1d-db9b-4c17-a39e-1335ae8761d4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.110121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.111177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.111667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.112148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.113191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:25.113736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.115349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.125373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0b351-8a30-4f27-9270-084e6e2f7c87 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.125888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 64/128] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:25.126951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.166217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.167639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.167639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.167828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.169080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.170564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.171503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:25.173873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 65/129] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:25 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:25.174974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.224840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:25.266528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a10ae-2afe-44b7-987f-00a73f9ed495 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.299869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760d559-e6bc-43e0-a310-18c5ae6a0e8c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.305508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.306464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.306947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.307426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.342105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.342105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.342105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.342446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.343851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.345274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.347126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 66/130] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:25 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:25.348908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.399839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.401756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.448625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:25.472028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dcc9f-6444-4daf-8873-75ecf40323a1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.474980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:25.524785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.525784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.526185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.526479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] PATH_INFO: `/v3/projects/19e74baba5a9416a98b86a80b07f4140` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.533651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] RBAC: Authorizing `identity:get_project(project_id=19e74baba5a9416a98b86a80b07f4140)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.535036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.536904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 67/131] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:25 2022] GET /identity/v3/projects/19e74baba5a9416a98b86a80b07f4140 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:25.537790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.538272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 65/132] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:25.549489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.683526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db22bcd-d5e3-486b-839f-061cccc89057 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.705245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcb753-af92-49c0-a120-a5cf41747d54 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.710838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:25.735616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.737263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.737556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.737908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] PATH_INFO: `/v3/projects/19e74baba5a9416a98b86a80b07f4140/users/1c3a6fffeecf41fb83087b2815dfe4b8/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.741115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=19e74baba5a9416a98b86a80b07f4140, user_id=1c3a6fffeecf41fb83087b2815dfe4b8, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.743271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.755923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3e74a-cf3d-4bb9-9944-e3693bc75229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.757500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3e74a-cf3d-4bb9-9944-e3693bc75229 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.758014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3e74a-cf3d-4bb9-9944-e3693bc75229 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.758591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3e74a-cf3d-4bb9-9944-e3693bc75229 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.760326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3e74a-cf3d-4bb9-9944-e3693bc75229 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.762423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3e74a-cf3d-4bb9-9944-e3693bc75229 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.764567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 68/133] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:25 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:25.764705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 66/134] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:25 2022] PUT /identity/v3/projects/19e74baba5a9416a98b86a80b07f4140/users/1c3a6fffeecf41fb83087b2815dfe4b8/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:25.768566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.771246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935699-1cd1-4752-b986-890d7791001d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.771618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935699-1cd1-4752-b986-890d7791001d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.771972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935699-1cd1-4752-b986-890d7791001d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.773811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 69/135] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:25.774437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935699-1cd1-4752-b986-890d7791001d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:25.853168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:25.910136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95823847-d450-49b6-97ac-9199e030adc6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.955029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc4f5b-c80c-488f-8c48-7268e08ac22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:25.956409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc4f5b-c80c-488f-8c48-7268e08ac22b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.957179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc4f5b-c80c-488f-8c48-7268e08ac22b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.957662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc4f5b-c80c-488f-8c48-7268e08ac22b None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.959255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc4f5b-c80c-488f-8c48-7268e08ac22b None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:25.961218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc4f5b-c80c-488f-8c48-7268e08ac22b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:25.961785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935699-1cd1-4752-b986-890d7791001d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:25.963789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 67/136] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:25 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:25.965275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:25.965775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:25.966254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:25.990265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:25.994702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:26.036781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:26.038110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.038633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.038927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276/users/0802dbaa2668431bbb6defab89b49e81/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.041026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e19f2b19b0472c87c1688310368276, user_id=0802dbaa2668431bbb6defab89b49e81, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:26.043399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:26.072484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 70/137] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:25 2022] PUT /identity/v3/projects/f7e19f2b19b0472c87c1688310368276/users/0802dbaa2668431bbb6defab89b49e81/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:26.072484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:26.102212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:26.134487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 68/138] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:26.168437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:26.214003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:26.226425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70447ad7-a2b2-4698-8eba-f257cbedf832 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:26.266659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:26.267739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.268218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.268725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.270387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] RBAC: Adding query filter params (role.id=625662ea5ad34314b41969de27beb3c5, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=108478f2d096419c908833ecfac69803) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:26.270926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:26.272714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364fd3e-2542-4df5-abb2-e53f9aaf0f65 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:26.280570 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 71/139] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:26 2022] GET /identity/v3/role_assignments?user.id=108478f2d096419c908833ecfac69803&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=625662ea5ad34314b41969de27beb3c5 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:26.282397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e9122-1ca6-46bb-a9f4-5605d6595c40 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.282937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e9122-1ca6-46bb-a9f4-5605d6595c40 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.283431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e9122-1ca6-46bb-a9f4-5605d6595c40 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.286009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 72/140] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:26.299681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.300512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.301003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.330616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:26.332927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:26.426503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c2f77-31cf-45bf-9bbe-d210c137d0f6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:26.453832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 73/141] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:26.468933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:26.470119 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.470615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.471087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.484322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:26.495025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:26.495897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:26.496656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:26.499203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:26.499913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 69/142] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:26.506802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:26.556319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:26.669888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184f422-afbd-4401-9182-11e7fd095cc0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:26.734305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:26.747521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882d8b-3de3-44d6-b066-da56b10854f8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:26.765124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:26.765820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.766115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.766398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.767171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:26.767480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:26.768919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:26.788124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 70/143] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:26.793817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:26.831665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:26.832848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:26.833140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:26.833422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:26.834696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:26.836188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:26.838251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 74/144] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:26.845985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:27.041019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbadffb-72fc-4916-a4bb-9de8c632ec3d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:27.121470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b9972-0dc2-45be-b2f1-7259c9d7c9e3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:27.124258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:27.130753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:27.131554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:27.131844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:27.132126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:27.132892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:27.135013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:27.190584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:27.192586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:27.193086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:27.193562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:27.197052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:27.198772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:27.202951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 71/145] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:26 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:27.204360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 75/146] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:26 2022] POST /identity/v3/users => generated 283 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:27.208891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:27.365428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6f79-fffd-4e05-bdd8-4d51acc48ca5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:27.408329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:27.409033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:27.409318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:27.410173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:27.463910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:27.502966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:27.504534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:27.627550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 72/147] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:27.633576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:27.669576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:27.779688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16f5b0-451c-4268-b2c8-4febf730671b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:27.820748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38116ca-f80d-4d99-a4a2-9ac132eba023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:27.821988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38116ca-f80d-4d99-a4a2-9ac132eba023 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:27.822494 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38116ca-f80d-4d99-a4a2-9ac132eba023 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:27.822976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38116ca-f80d-4d99-a4a2-9ac132eba023 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:27.824356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38116ca-f80d-4d99-a4a2-9ac132eba023 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:27.826028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38116ca-f80d-4d99-a4a2-9ac132eba023 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:27.828216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 76/148] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:27 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:27.834786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:27.937205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:27.978744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca36a9-4732-49c2-90aa-c9ffffed6d3f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:28.015975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:28.016728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.017012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.017288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:28.018136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=1c3a6fffeecf41fb83087b2815dfe4b8) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:28.018446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:28.020573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c3de6-b4d2-44c4-a5c0-2bcb9ea865aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:28.028535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 73/149] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:27 2022] GET /identity/v3/role_assignments?user.id=1c3a6fffeecf41fb83087b2815dfe4b8&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:28.042785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc75953-d08d-4097-a3e8-a6422d9a3191 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.043397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc75953-d08d-4097-a3e8-a6422d9a3191 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.043881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc75953-d08d-4097-a3e8-a6422d9a3191 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:28.045654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 77/150] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:28.058530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.059160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.059627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:28.078755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:28.080282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:28.176279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 74/151] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:28.219175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc75953-d08d-4097-a3e8-a6422d9a3191 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:28.325519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:28.348059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc75953-d08d-4097-a3e8-a6422d9a3191 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:28.390819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:28.392023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.392529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.392999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:28.407333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:28.415933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:28.416907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:28.417659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:28.420071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 78/152] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:28.426385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:28.511442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:28.563773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bdb72-9a8c-4330-a974-f0d5cbd6a068 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:28.610261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:28.611346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.611821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.612283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:28.613894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:28.614427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:28.616195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:28.626521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 75/153] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:28.633056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:28.781812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c4bd7-cda4-42d8-8a76-14ca9ca43e3d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:28.828435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbac4a-2fce-4fbb-94b1-a3153f13892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:28.829394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbac4a-2fce-4fbb-94b1-a3153f13892b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.829684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbac4a-2fce-4fbb-94b1-a3153f13892b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.829967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbac4a-2fce-4fbb-94b1-a3153f13892b None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:28.831326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbac4a-2fce-4fbb-94b1-a3153f13892b None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:28.832911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbac4a-2fce-4fbb-94b1-a3153f13892b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:28.834988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 79/154] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:28 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:28.840751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:28.953032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cadfb-9342-4ce2-83c1-0cb61d495f30 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:28.981101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:28.995120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:28.995805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:28.996093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:28.996374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.030337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f614-d7ff-4457-b9f1-b12f88a1792d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.030715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f614-d7ff-4457-b9f1-b12f88a1792d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.030938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f614-d7ff-4457-b9f1-b12f88a1792d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.032974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 80/155] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:29.052473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.056412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.056412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.092942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:29.094188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:29.118435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:29.119933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:29.124531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:29.273824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 81/156] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:29.290629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 76/157] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:29.301573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:29.313912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:29.354725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:29.471534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d1138-cd3f-42eb-b214-75262c28c9be None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:29.554539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55342023-0c0b-4e61-8dce-e887608cba3f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:29.573747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29940a-bea8-4c21-8d15-05732af2e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:29.574573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29940a-bea8-4c21-8d15-05732af2e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.574860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29940a-bea8-4c21-8d15-05732af2e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.575140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29940a-bea8-4c21-8d15-05732af2e1e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.607713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e66d75-8d45-4cab-9940-fb173f20c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:29.608671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e66d75-8d45-4cab-9940-fb173f20c683 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.608958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e66d75-8d45-4cab-9940-fb173f20c683 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.609239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e66d75-8d45-4cab-9940-fb173f20c683 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.610473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e66d75-8d45-4cab-9940-fb173f20c683 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:29.612010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e66d75-8d45-4cab-9940-fb173f20c683 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:29.614147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 82/158] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:29 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:29.614592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29940a-bea8-4c21-8d15-05732af2e1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:29.615268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606651e-702c-4f20-9b2d-9acb9dc6ef49 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.615561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606651e-702c-4f20-9b2d-9acb9dc6ef49 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.615839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606651e-702c-4f20-9b2d-9acb9dc6ef49 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.617249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 83/159] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:29.621295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4ca8-293b-4d17-9a5f-d2389f73a751 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.621577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4ca8-293b-4d17-9a5f-d2389f73a751 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.621857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4ca8-293b-4d17-9a5f-d2389f73a751 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.623321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 84/160] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6d29940a-bea8-4c21-8d15-05732af2e1e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:29.623489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:29.625440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:29.627437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 77/161] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:29.630463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4ca8-293b-4d17-9a5f-d2389f73a751 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:29.637950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e6dec-c4ff-4396-a618-628c0a3ad017 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.638219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e6dec-c4ff-4396-a618-628c0a3ad017 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.638504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e6dec-c4ff-4396-a618-628c0a3ad017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.654130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:29.669289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f29e6dec-c4ff-4396-a618-628c0a3ad017 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:29.676846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f29e6dec-c4ff-4396-a618-628c0a3ad017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:29.747432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4ca8-293b-4d17-9a5f-d2389f73a751 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:29.792325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:29.793278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.793278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.793563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.796295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] RBAC: Adding query filter params (role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=1c3a6fffeecf41fb83087b2815dfe4b8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:29.796295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:29.796295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:29.812340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 85/162] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:29 2022] GET /identity/v3/role_assignments?user.id=1c3a6fffeecf41fb83087b2815dfe4b8&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=c41b65addaec48bfb92f135a2a724132 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:29.813239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:29.839135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 78/163] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:29.840459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:29.840749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:29.841019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:29.883345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:29.884698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:29.907575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:30.015688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 79/164] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:30.016598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:30.072490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:30.141222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbaba2d-4e0b-436a-92b1-9ac4a71fa531 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:30.224218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95d58c-87e8-4447-a6fa-4ea45270627d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:30.236707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:30.238386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:30.238875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:30.239320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:30.240265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:30.240758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:30.242429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:30.262627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 86/165] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:30.263928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:30.281028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:30.281844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:30.282127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:30.282406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:30.308016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:30.317931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:30.318835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 80/166] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:30.320017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:30.351684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:30.486712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3f125-9b31-4600-a936-c0136d0faecb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:30.504147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c5e34-e634-4aa7-9f12-8ce10d9f04e1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:30.529243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:30.530132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:30.530413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:30.530687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] PATH_INFO: `/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:30.532089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] RBAC: Authorizing `identity:get_user(user_id=1c3a6fffeecf41fb83087b2815dfe4b8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:30.533597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:30.535795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 81/167] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:30 2022] GET /identity/v3/users/1c3a6fffeecf41fb83087b2815dfe4b8 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:30.542156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:30.577659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:30.578481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:30.578766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:30.579039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:30.598424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:30.607570 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:30.618029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:30.618927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 87/168] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:30.621162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:30.745367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddecf4-7f04-41b3-9e22-0e6bd459fd36 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:30.748401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:30.783589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3d5b3-6c8c-478f-9c4f-21bbbf3e4a37 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:30.809376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:30.810054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:30.810348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:30.810623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:30.858214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:30.858917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:30.859204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:30.859480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:30.860363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:30.860695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:30.863114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:30.876113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 82/169] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:30.877511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:30.884100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:30.887041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:30.888855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.056415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f5e1e-e41a-4286-a17d-ea90471a648b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.076730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 88/170] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:31.076730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.126581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.127618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.128095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.129134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.130099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:31.130621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.140415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.146533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:31.158066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 83/171] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:31.159347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.208080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:31.213022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddeed62-5ae5-49b6-b9da-f2292797eee4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.297102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.297997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.298280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.298555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] PATH_INFO: `/v3/users/108478f2d096419c908833ecfac69803` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.304844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] RBAC: Authorizing `identity:get_user(user_id=108478f2d096419c908833ecfac69803)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.306324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.308290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 89/172] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/users/108478f2d096419c908833ecfac69803 => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:31.308886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.349141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f76048-0209-4cb5-9443-1cfa8b27f061 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.391478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.392270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.392625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.393020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] PATH_INFO: `/v3/projects/19e74baba5a9416a98b86a80b07f4140` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.394367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] RBAC: Authorizing `identity:get_project(project_id=19e74baba5a9416a98b86a80b07f4140)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.395840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.402664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 84/173] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/projects/19e74baba5a9416a98b86a80b07f4140 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:31.402664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.441916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80073e83-63cb-4b46-9a54-5542c5d3ceb0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.480738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:31.483726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124768db-d732-48cc-a266-4cbf0ed2ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.484930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124768db-d732-48cc-a266-4cbf0ed2ee34 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.485313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124768db-d732-48cc-a266-4cbf0ed2ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.485591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124768db-d732-48cc-a266-4cbf0ed2ee34 None admin] PATH_INFO: `/v3/users/0802dbaa2668431bbb6defab89b49e81` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.487058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124768db-d732-48cc-a266-4cbf0ed2ee34 None admin] RBAC: Authorizing `identity:get_user(user_id=0802dbaa2668431bbb6defab89b49e81)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.490073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124768db-d732-48cc-a266-4cbf0ed2ee34 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.491726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 90/174] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/users/0802dbaa2668431bbb6defab89b49e81 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:31.494547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2cd46-be83-4d8c-a9e6-ceccb9c394a3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.495322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2cd46-be83-4d8c-a9e6-ceccb9c394a3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.496133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2cd46-be83-4d8c-a9e6-ceccb9c394a3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.498437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 91/175] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:31.500725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96eecdf-2c19-45cb-b9d2-263f8a69065b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.501262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96eecdf-2c19-45cb-b9d2-263f8a69065b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.501738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96eecdf-2c19-45cb-b9d2-263f8a69065b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.503890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 92/176] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:31.504950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96eecdf-2c19-45cb-b9d2-263f8a69065b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.530527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74d03c-abbf-4caa-b512-51aaa4dd0adf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.541976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:31.566619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.567521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.567800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.568068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.569586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.571079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.572967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 85/177] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:31.577652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.618222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96eecdf-2c19-45cb-b9d2-263f8a69065b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.653598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.654165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.654448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.654719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.655555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=19e74baba5a9416a98b86a80b07f4140, user.id=1c3a6fffeecf41fb83087b2815dfe4b8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:31.655859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.657371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44ae20-c664-4811-a623-f7d4c04f6488 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.663524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 93/178] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/role_assignments?user.id=1c3a6fffeecf41fb83087b2815dfe4b8&scope.project.id=19e74baba5a9416a98b86a80b07f4140&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:31.676728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a9bd6-e5b1-44bc-9191-6a6633526688 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.676728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a9bd6-e5b1-44bc-9191-6a6633526688 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.676728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a9bd6-e5b1-44bc-9191-6a6633526688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.684576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de261f8-12ca-494f-b0dc-65d50a80af34 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:31.686103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-061a9bd6-e5b1-44bc-9191-6a6633526688 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:31.688615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-061a9bd6-e5b1-44bc-9191-6a6633526688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:31.726207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:31.726671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.726958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.727367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.730644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:31.774384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 94/179] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:31.775962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c306e-bbdd-4513-986f-e9328b11a4c5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.776416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c306e-bbdd-4513-986f-e9328b11a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.776829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c306e-bbdd-4513-986f-e9328b11a4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.796629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ff5c306e-bbdd-4513-986f-e9328b11a4c5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:31.798447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ff5c306e-bbdd-4513-986f-e9328b11a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:31.820267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:31.821791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:31.826941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:31.890275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 95/180] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:31.891799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77dc6-8733-4c2d-b85d-e97519f5363b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:31.892085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77dc6-8733-4c2d-b85d-e97519f5363b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:31.892360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77dc6-8733-4c2d-b85d-e97519f5363b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:31.893970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 96/181] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:31.894525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77dc6-8733-4c2d-b85d-e97519f5363b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.946066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 86/182] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:31.946983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:31.980368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.007951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77dc6-8733-4c2d-b85d-e97519f5363b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.042395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc343d-5735-4780-8e52-1b563cfe7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.043293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc343d-5735-4780-8e52-1b563cfe7ad8 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.043573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc343d-5735-4780-8e52-1b563cfe7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.043855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc343d-5735-4780-8e52-1b563cfe7ad8 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276/users/108478f2d096419c908833ecfac69803/roles/625662ea5ad34314b41969de27beb3c5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.045896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc343d-5735-4780-8e52-1b563cfe7ad8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e19f2b19b0472c87c1688310368276, user_id=108478f2d096419c908833ecfac69803, role_id=625662ea5ad34314b41969de27beb3c5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.046593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07431d2-36b3-4f60-8ba1-ed944ec9c39c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.047371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc343d-5735-4780-8e52-1b563cfe7ad8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.062277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 97/183] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:31 2022] PUT /identity/v3/projects/f7e19f2b19b0472c87c1688310368276/users/108478f2d096419c908833ecfac69803/roles/625662ea5ad34314b41969de27beb3c5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:32.063621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.063915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.064191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.084934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.085737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.086017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.086287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.095045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:32.096310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:32.100833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:32.108631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:32.109611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:32.110372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 87/184] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:32.111017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.120126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.192313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 98/185] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:32.196892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.241272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.249130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e8acf-3f11-4730-a225-d5c31353bd7b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.294992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.295829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.296108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.296379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.307276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:32.315485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:32.316583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 88/186] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:32.322325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.329107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49093ad6-bd5a-4306-b16e-a85be4ef04d1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.367252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.368573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.368857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.369144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.370352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.371799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.373749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 99/187] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:32.374443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.397983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.436412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30b6f7-cf02-4734-9eae-7c98bbcb6413 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.476582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.477195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.477554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.477836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.478621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:32.480157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.481692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.493980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65442e54-ea09-44b8-b73c-3915b953f316 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.496682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 89/188] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:32.497410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.518742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.537346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.538067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.538352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.538626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.539841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.541325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.542979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 100/189] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:32.543752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.618097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482a8e3-76d3-49e1-8a29-4a0d91031f5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.636846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.654972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.655525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523beb22-76f3-431a-9f2a-f16424e4595e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.655630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.655913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.656186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.656972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:32.657298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.658678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.667666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 90/190] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:32.668593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.707580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.709016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.709744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.710212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.711395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=0802dbaa2668431bbb6defab89b49e81) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:32.712224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.714041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.722259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 101/191] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/role_assignments?user.id=0802dbaa2668431bbb6defab89b49e81&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:32.723323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.812183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:32.842050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd770a8-8250-4535-bad8-46378e8be948 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:32.886887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:32.888204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:32.888812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:32.889813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] PATH_INFO: `/v3/users/a76cd249767840b58a4b94babbbf4daf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:32.908729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] RBAC: Authorizing `identity:get_user(user_id=a76cd249767840b58a4b94babbbf4daf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:32.910597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:32.912929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 91/192] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/users/a76cd249767840b58a4b94babbbf4daf => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:32.913927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:32.962219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ce312-af14-4feb-8cf0-38548b9a2013 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:33.012823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:33.014021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:33.014502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:33.014995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:33.015931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:33.019296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:33.056560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 102/193] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:32 2022] POST /identity/v3/users => generated 297 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:33.058065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:33.062718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e724b-5b75-47b3-93c7-bc31b869455d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:33.063680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:33.169157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:33.170540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:33.171080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:33.171549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] PATH_INFO: `/v3/users/a76cd249767840b58a4b94babbbf4daf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:33.173418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] RBAC: Authorizing `identity:get_user(user_id=a76cd249767840b58a4b94babbbf4daf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:33.175863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:33.178008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 92/194] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:32 2022] GET /identity/v3/users/a76cd249767840b58a4b94babbbf4daf => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:33.185350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:33.193735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:33.274709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60e42-f0f8-4d76-80ff-156d7f255d64 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:33.316850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:33.317910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:33.318448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:33.318934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:33.345048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9f06f-60d1-48fb-b42e-78886f985d2a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:33.386875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:33.387554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:33.387840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:33.388120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:33.396815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:33.398339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:33.402261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:33.490436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:33.491936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:33.495813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:33.532655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 103/195] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:33.537563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:33.671372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 93/196] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:33.677125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:33.770995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8d1a2-08a5-4e8c-9586-def0f72982ba None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:33.774387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:33.811438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88668c5-a121-401d-9858-0786d1a02dcb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:33.819985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:33.850846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:33.852308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:33.853355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:33.854255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:33.860270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:33.861229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:33.861518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:33.861773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:33.861873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:33.863083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:33.863852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:33.864572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:33.866101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 104/197] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:33 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:33.866466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 94/198] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:33 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:33.871973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:33.872514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:34.021716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b454ad-b0b7-457d-acf5-1dba7de34aa5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:34.057833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:34.058531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:34.058814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:34.059093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:34.059938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=a76cd249767840b58a4b94babbbf4daf) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:34.060242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:34.061724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590322-c358-43da-9a97-6d6d71fe687d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:34.064871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b309f6a-5e28-4dfc-8d34-30c44dbd656a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:34.068924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 105/199] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:33 2022] GET /identity/v3/role_assignments?user.id=a76cd249767840b58a4b94babbbf4daf&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:34.135658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:34.136345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:34.136656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:34.136933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:34.137804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] RBAC: Adding query filter params (role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=a76cd249767840b58a4b94babbbf4daf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:34.138113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:34.139632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a00739-a405-40f6-b712-ab3e723f0923 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:34.153314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 95/200] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:33 2022] GET /identity/v3/role_assignments?user.id=a76cd249767840b58a4b94babbbf4daf&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=c41b65addaec48bfb92f135a2a724132 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:34.247332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9d79-d189-4809-9992-dc322ee61746 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:34.247791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9d79-d189-4809-9992-dc322ee61746 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:34.248082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9d79-d189-4809-9992-dc322ee61746 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:34.249836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 106/201] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:34.256184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:34.256497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:34.256856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:34.285566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:34.286684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:34.344714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:34.395486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 96/202] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:34.432032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9d79-d189-4809-9992-dc322ee61746 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:34.483629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:34.565440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9d79-d189-4809-9992-dc322ee61746 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:34.609137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:34.610018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:34.610306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:34.610585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:34.625305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:34.633325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:34.634989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:34.636593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 107/203] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:34.645517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:34.810587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d82f7-1b8b-40e1-8ccf-27400a1afbea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:34.857368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:34.857942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:34.858201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:34.858490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:34.859253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:34.859556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:34.860987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:34.877854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 97/204] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:34.887478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:35.023165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:35.029207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986842-6ffc-4443-8f72-be565c71ea5f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:35.077494 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:35.078391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:35.078669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:35.078948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] PATH_INFO: `/v3/users/108478f2d096419c908833ecfac69803` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:35.085974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] RBAC: Authorizing `identity:get_user(user_id=108478f2d096419c908833ecfac69803)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:35.087450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:35.089157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 108/205] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:34 2022] GET /identity/v3/users/108478f2d096419c908833ecfac69803 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:35.094840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:35.148790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:35.308706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63ccd9-fd7f-4497-a427-6106531fb1c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:35.355464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61535f44-5887-4d33-96c5-7d5141b6a356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:35.356117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61535f44-5887-4d33-96c5-7d5141b6a356 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:35.356420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61535f44-5887-4d33-96c5-7d5141b6a356 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:35.356695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61535f44-5887-4d33-96c5-7d5141b6a356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:35.455116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61535f44-5887-4d33-96c5-7d5141b6a356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:35.456571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61535f44-5887-4d33-96c5-7d5141b6a356 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:35.497929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:35.584635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 98/206] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:35.590247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:35.761659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d642d6-9e62-40f4-8f59-d3b615da594d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:35.774064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9fa8-923b-4f47-8580-d33a19012579 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:35.774573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9fa8-923b-4f47-8580-d33a19012579 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:35.774645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9fa8-923b-4f47-8580-d33a19012579 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:35.776221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 99/207] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:35.786328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b314ed-65af-4a7b-baef-85131d435a0e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:35.786623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b314ed-65af-4a7b-baef-85131d435a0e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:35.786893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b314ed-65af-4a7b-baef-85131d435a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:35.803929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:35.804948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:35.805238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:35.805512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:35.806716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:35.808289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:35.810268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 109/208] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:35 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:35.815782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:35.828933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-62b314ed-65af-4a7b-baef-85131d435a0e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:35.830155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-62b314ed-65af-4a7b-baef-85131d435a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:35.873013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:35.928577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb0ce7-3f2c-4eb0-8dc2-c90ebbfc21b4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:35.966781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:35.967742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:35.968201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:35.968678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:35.969721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] RBAC: Adding query filter params (role.id=625662ea5ad34314b41969de27beb3c5, scope.project.id=f7e19f2b19b0472c87c1688310368276, user.id=108478f2d096419c908833ecfac69803) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:35.970210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:35.971823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:35.980348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 110/209] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:35 2022] GET /identity/v3/role_assignments?user.id=108478f2d096419c908833ecfac69803&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=625662ea5ad34314b41969de27beb3c5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:36.010207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 100/210] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:36.037041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:36.153640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad916a0-51cd-418a-8193-9423b647f812 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:36.156750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:36.164428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab62e5-75ec-4722-b392-f292abe3aecc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.164768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab62e5-75ec-4722-b392-f292abe3aecc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.165224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab62e5-75ec-4722-b392-f292abe3aecc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.166539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 101/211] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:36.175337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.175866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.176338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.199858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:36.201307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:36.228066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:36.229189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.229682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.230136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.247328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:36.258141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:36.259055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:36.259811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 111/212] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:36.265438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:36.276208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:36.296484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 102/213] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:36.322343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:36.402821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a04384-04f1-4d03-8603-9fc68f9c351c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:36.457754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:36.458437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.458816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.459094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.459960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:36.460265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:36.461688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:36.466892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08275288-5694-42b0-8250-14e15b3f9004 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:36.471615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 112/214] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:36.477763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:36.501997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:36.524720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:36.525590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.525873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.526148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.527374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:36.529057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:36.530762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 103/215] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:36.542165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:36.608953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ea144-3304-405e-9131-f909c43c2b8d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:36.610400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:36.654331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:36.655742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.656322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.656830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] PATH_INFO: `/v3/users/94cc2e0008e74b79bd20287b43bdb1f1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.663212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba097a2-f541-43f1-9bdb-73142602ec43 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:36.675727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] RBAC: Authorizing `identity:get_user(user_id=94cc2e0008e74b79bd20287b43bdb1f1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:36.677925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:36.680160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 113/216] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:36 2022] GET /identity/v3/users/94cc2e0008e74b79bd20287b43bdb1f1 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:36.688683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:36.709073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4e938-f68c-432d-8316-45476e4d2c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:36.709815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e938-f68c-432d-8316-45476e4d2c91 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.710100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e938-f68c-432d-8316-45476e4d2c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.710376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e938-f68c-432d-8316-45476e4d2c91 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.711221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4e938-f68c-432d-8316-45476e4d2c91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:36.712792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4e938-f68c-432d-8316-45476e4d2c91 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:36.739498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 104/217] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:36 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:36.797930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:36.829585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb49788-b828-4292-acc3-67f8a38363a1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:36.846947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b5f99-37a6-4e01-810f-b066ef906e95 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.847503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b5f99-37a6-4e01-810f-b066ef906e95 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.847577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b5f99-37a6-4e01-810f-b066ef906e95 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.849344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 105/218] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:36.854976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.855338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.855588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.874739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:36.875900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:36.880070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:36.880839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:36.881124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:36.881402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:36.967839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 106/219] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:36.971823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:36.974660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:36.993651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:37.069956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:37.086709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:37.128450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 114/220] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 13:46:37.137039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:37.170161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e028c5-023a-426b-8a13-33043cf13b27 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:37.214962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:37.216028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.216310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.216621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.229996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:37.239110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:37.240023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:37.240833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 107/221] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:37.248195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:37.292786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b5406-f7b0-476f-9ce6-7792b65bf584 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:37.360086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790eac1-bcc7-4067-8427-c19fc97af03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:37.362075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790eac1-bcc7-4067-8427-c19fc97af03e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.362652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790eac1-bcc7-4067-8427-c19fc97af03e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.363131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790eac1-bcc7-4067-8427-c19fc97af03e None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.365702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790eac1-bcc7-4067-8427-c19fc97af03e None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:37.368968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790eac1-bcc7-4067-8427-c19fc97af03e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:37.377778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 115/222] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:37 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:37.380690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ad937-953f-4011-8bbf-7d44d5211cac None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.381017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ad937-953f-4011-8bbf-7d44d5211cac None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.381545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ad937-953f-4011-8bbf-7d44d5211cac None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.384529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 116/223] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:37.390493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ad937-953f-4011-8bbf-7d44d5211cac None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:37.427277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aef4e-22d4-4a2a-a26b-d43bbd23bb6c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:37.429779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:37.479027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:37.480203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.480727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.481208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.482580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:37.483108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:37.486505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c73d2e-a7ae-495f-bc45-9cc315e7eb3d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:37.495948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:37.504721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 108/224] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:37.507377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.507861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.508337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.541659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:37.543894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:37.606597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ad937-953f-4011-8bbf-7d44d5211cac None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:37.669029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:37.670447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.671148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.671884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.673686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=94cc2e0008e74b79bd20287b43bdb1f1) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:37.674482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:37.677607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:37.685845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 109/225] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:37.687305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:37.688891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 117/226] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:37 2022] GET /identity/v3/role_assignments?user.id=94cc2e0008e74b79bd20287b43bdb1f1&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:37.739169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:37.885043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:37.901820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34036ec0-7af0-492a-ad76-52a3e6b5a536 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:37.988346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:37.990338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:37.990620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:37.990894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] PATH_INFO: `/v3/users/a76cd249767840b58a4b94babbbf4daf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:37.995323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] RBAC: Authorizing `identity:get_user(user_id=a76cd249767840b58a4b94babbbf4daf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:38.001925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:38.004480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 110/227] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:37 2022] GET /identity/v3/users/a76cd249767840b58a4b94babbbf4daf => generated 283 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:38.009833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:38.076569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f633-00e8-453a-a105-bba18d16804e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:38.088354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:38.118986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:38.119803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:38.120077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:38.120346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:38.133425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:38.142280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:38.143152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:38.143961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:38.147605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 118/228] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:38.153280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:38.267328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218992fc-5a07-4815-8161-c5ac91072f3b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:38.309163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d248c9-db51-4c19-b5fd-498595718068 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:38.326972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:38.327851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:38.328132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:38.328422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:38.331043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:38.332595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:38.334511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 111/229] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:38 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:38.352355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:38.357048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:38.357971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:38.358439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:38.358909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:38.359853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:38.360363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:38.362046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:38.389391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 119/230] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:38.400709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:38.408747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:38.484132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa22eb-46a0-49c8-b358-4400a36ac384 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:38.526205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:38.527252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:38.527535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:38.527809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/a76cd249767840b58a4b94babbbf4daf/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:38.530013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] RBAC: Authorizing `identity:create_grant(project_id=181d8587bb7b465483b60f3ec85eefda, user_id=a76cd249767840b58a4b94babbbf4daf, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:38.531813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:38.538162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:38.546744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 112/231] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:38 2022] PUT /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/a76cd249767840b58a4b94babbbf4daf/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:38.667445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86a773-0982-4412-9e09-ec6ded81938b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:38.727702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:38.728922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:38.729350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:38.729789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] PATH_INFO: `/v3/users/a76cd249767840b58a4b94babbbf4daf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:38.731288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] RBAC: Authorizing `identity:get_user(user_id=a76cd249767840b58a4b94babbbf4daf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:38.732964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:38.734871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 120/232] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:38 2022] GET /identity/v3/users/a76cd249767840b58a4b94babbbf4daf => generated 283 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:38.741390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:38.889115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:38.893303 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfdaec-6d93-407a-901a-572593876ecf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:38.939658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e410c00-8206-4cb0-b3ba-91fa2dd1a636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:38.940566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e410c00-8206-4cb0-b3ba-91fa2dd1a636 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:38.941535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e410c00-8206-4cb0-b3ba-91fa2dd1a636 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:38.941966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e410c00-8206-4cb0-b3ba-91fa2dd1a636 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:38.943293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e410c00-8206-4cb0-b3ba-91fa2dd1a636 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:38.944767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e410c00-8206-4cb0-b3ba-91fa2dd1a636 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:38.946894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 113/233] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:38 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:38.953021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:39.118408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:39.128854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65b813-73d7-4b15-92cc-8718ac723e87 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:39.189837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78efd52-5338-422c-b246-f5de06ea16ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:39.193035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78efd52-5338-422c-b246-f5de06ea16ff None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.193035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78efd52-5338-422c-b246-f5de06ea16ff None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.193035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78efd52-5338-422c-b246-f5de06ea16ff None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/a76cd249767840b58a4b94babbbf4daf/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.195276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78efd52-5338-422c-b246-f5de06ea16ff None admin] RBAC: Authorizing `identity:create_grant(project_id=181d8587bb7b465483b60f3ec85eefda, user_id=a76cd249767840b58a4b94babbbf4daf, role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:39.196773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78efd52-5338-422c-b246-f5de06ea16ff None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:39.224650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 121/234] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:38 2022] PUT /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/a76cd249767840b58a4b94babbbf4daf/roles/c41b65addaec48bfb92f135a2a724132 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:39.305107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442ade5-0bc5-4118-8446-a935ea52e28d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.305725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442ade5-0bc5-4118-8446-a935ea52e28d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.306198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442ade5-0bc5-4118-8446-a935ea52e28d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.308102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 114/235] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:39.314215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.314510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.314795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.342446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:39.343200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c302e-3f4e-400d-9816-4296b5c8decb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.343688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:39.343958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c302e-3f4e-400d-9816-4296b5c8decb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.344456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c302e-3f4e-400d-9816-4296b5c8decb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.346032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 115/236] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:39.351891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.352204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.352465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.380458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:39.381696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:39.455148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 122/237] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:39.480234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:39.504359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 116/238] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:39.540731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:39.613018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:39.632916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2ea65-0865-40c9-97b2-d5f4ef701a85 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:39.696587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:39.714263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac0001-c179-4092-89a7-207f55847100 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:39.734022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:39.735224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.735802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.736272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.760073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:39.781236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:39.782149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:39.782898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 123/239] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:39.788426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:39.798634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:39.800193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:39.800936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:39.801392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:39.802955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:39.809431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:39.811160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 117/240] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:39.818791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:39.965636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1232d7-b64e-44e0-9fdc-1b98426fa6d4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:39.996040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49768c1-6ea9-4c91-89c2-2201185efa7d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:40.028675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:40.029770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.030272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.030738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.031737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:40.033428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:40.061166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 118/241] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:39 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:40.065097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:40.066815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.067365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.067889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.069083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:40.069682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:40.071473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:40.082886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 124/242] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:40.089627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:40.113503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:40.238527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315940e-95e5-4207-8050-d1c60f6b37f5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:40.276952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cb9c3-7d6a-4f28-b237-af83d9ef8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:40.277836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cb9c3-7d6a-4f28-b237-af83d9ef8575 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.278119 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cb9c3-7d6a-4f28-b237-af83d9ef8575 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.278394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cb9c3-7d6a-4f28-b237-af83d9ef8575 None admin] PATH_INFO: `/v3/users/ef993511e71041d59010fbe6b805617a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.292305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cb9c3-7d6a-4f28-b237-af83d9ef8575 None admin] RBAC: Authorizing `identity:get_user(user_id=ef993511e71041d59010fbe6b805617a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:40.293820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cb9c3-7d6a-4f28-b237-af83d9ef8575 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:40.295862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 119/243] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:40 2022] GET /identity/v3/users/ef993511e71041d59010fbe6b805617a => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:40.301500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:40.317083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:40.352431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50b3ca-2943-4675-b9bf-4dfa1997ef76 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.352768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50b3ca-2943-4675-b9bf-4dfa1997ef76 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.353000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50b3ca-2943-4675-b9bf-4dfa1997ef76 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.354637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 120/244] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:40.364048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.364330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.364729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.389151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:40.391008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:40.429943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224a199-ba26-449b-86ed-5f6d0bbc39d2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:40.480600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 121/245] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:40.480784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:40.483608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:40.483608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.483608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.483608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.512183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:40.592170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:40.595862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:40.612237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:40.651617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0aec0-b2df-49f9-80cb-a0be70dea109 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:40.690315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:40.691704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.691984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.692257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.703280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:40.711609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:40.712528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:40.717146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 122/246] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:40.724710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:40.739113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 125/247] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 13:46:40.745105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:40.861899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61b761-9857-4563-9ff3-61297597d175 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:40.874956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:40.885804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:40.907458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:40.908879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:40.909878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:40.910351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:40.912143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:40.914031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:40.916085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 126/248] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:40 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:40.922061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:41.010605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a332074-2031-447d-9a01-6113553864b4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:41.125229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:41.125846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.126134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.126414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.127194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:41.127502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:41.129503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e254a-2a98-4e50-ae2d-8fdabfa1ebb0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:41.140144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45382e51-0272-40b2-8b1a-21b18718ccb1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:41.163390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 123/249] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:41.164906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39149c6-1c63-4caa-bab7-90c4c5442ba6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.165195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39149c6-1c63-4caa-bab7-90c4c5442ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.165475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39149c6-1c63-4caa-bab7-90c4c5442ba6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.166831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 124/250] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:41.176102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39149c6-1c63-4caa-bab7-90c4c5442ba6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:41.198775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:41.199493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.199776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.200053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.200945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] RBAC: Adding query filter params (role.id=625662ea5ad34314b41969de27beb3c5, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=ef993511e71041d59010fbe6b805617a) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:41.201282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:41.202738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576d58-7a7f-4ad1-8f6e-be36fbbfc4c0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:41.209836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 127/251] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:40 2022] GET /identity/v3/role_assignments?user.id=ef993511e71041d59010fbe6b805617a&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=625662ea5ad34314b41969de27beb3c5 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:41.211245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.217085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.217499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.248637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:41.249855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:41.367803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:41.397234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39149c6-1c63-4caa-bab7-90c4c5442ba6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:41.433416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 128/252] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:41.451603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:41.471740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:41.481238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:41.482097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.482380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.482657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] PATH_INFO: `/v3/users/94cc2e0008e74b79bd20287b43bdb1f1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.483961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] RBAC: Authorizing `identity:get_user(user_id=94cc2e0008e74b79bd20287b43bdb1f1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:41.485444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:41.487497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 125/253] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:41 2022] GET /identity/v3/users/94cc2e0008e74b79bd20287b43bdb1f1 => generated 297 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:41.493069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:41.656229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4d85-e046-42eb-b091-dc3d2c005c4a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:41.675159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c7479-29a7-43f8-b3d8-b66b6abe1686 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:41.738589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:41.739403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.739687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.739963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.748099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:41.749001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:41.749287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:41.749567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:41.751410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:41.753660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:41.755631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 126/254] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:41 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:41.761101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:41.761990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:41.770376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:41.771239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:41.772007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 129/255] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:41.781147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:41.845181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:41.875216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:41.932945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebccfa-8386-4b8c-86c2-af140e1514f9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:41.949283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410c353-5b56-431b-b684-70412c85efc4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:42.009349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:42.010320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.010605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.010884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/94cc2e0008e74b79bd20287b43bdb1f1/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.012645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:42.013362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.013642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.013920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.014738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:42.015046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:42.016608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d8f8a-e13e-49f6-b3b7-0b7bd8d77aab None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:42.021352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=181d8587bb7b465483b60f3ec85eefda, user_id=94cc2e0008e74b79bd20287b43bdb1f1, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:42.022859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:42.034424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 130/256] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:42.035795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679edfd4-b512-4eec-b2a0-b8b5e5b7dfd0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.036073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679edfd4-b512-4eec-b2a0-b8b5e5b7dfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.036357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679edfd4-b512-4eec-b2a0-b8b5e5b7dfd0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.037914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 131/257] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:42.051071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 127/258] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:41 2022] PUT /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/94cc2e0008e74b79bd20287b43bdb1f1/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:42.052199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:42.053157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.054010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.055069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.101941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:42.104356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:42.271542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 132/259] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:42.280030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fe715-0fd4-46b4-80be-f56293ce9e40 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:42.312765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:42.321935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:42.334528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:42.335647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.335930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.336316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] PATH_INFO: `/v3/users/a76cd249767840b58a4b94babbbf4daf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.337636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] RBAC: Authorizing `identity:get_user(user_id=a76cd249767840b58a4b94babbbf4daf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:42.341045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:42.343465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 128/260] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:42 2022] GET /identity/v3/users/a76cd249767840b58a4b94babbbf4daf => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:42.349748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:42.368944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:42.468653 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949cf7-4613-43da-8621-05427f0e088a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:42.485695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a27cec-bc12-49a6-a7bd-9465f125d741 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:42.506530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:42.507295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.507581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.507861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.542855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:42.543596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.543883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.544164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.565477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:42.576944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:42.579696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:42.582384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:42.584264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 133/261] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:42.592868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:42.596297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:42.603198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:42.759041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:42.799177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8eec5d-2d7f-4213-a5d3-5960d2e7db9f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:42.799988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 129/262] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 13:46:42.807496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:42.848575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:42.850265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:42.850725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:42.851173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:42.852132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:42.852639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:42.854956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:42.866177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 134/263] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:42.873501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:43.045570 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c9287-367d-48a1-8627-4000af020e75 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:43.070943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:43.093212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb77ee9-d6d7-46aa-8a7a-0d1dda5bcfa2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:43.145174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:43.146101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.146391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.146671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] PATH_INFO: `/v3/users/a76cd249767840b58a4b94babbbf4daf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.149594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=a76cd249767840b58a4b94babbbf4daf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:43.151117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:43.153289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 135/264] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:42 2022] GET /identity/v3/users/a76cd249767840b58a4b94babbbf4daf => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:43.165538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:43.188798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:43.189359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:43.190254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.190533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.191434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.193434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:43.195723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:43.203008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 130/265] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:42 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:43.209456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:43.348089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31302b7b-1bdd-4b3e-b624-a02e50d8e1c5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:43.413989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21198aed-683c-46bb-bacb-fd264b0fe244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:43.414746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21198aed-683c-46bb-bacb-fd264b0fe244 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.415048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21198aed-683c-46bb-bacb-fd264b0fe244 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.415174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21198aed-683c-46bb-bacb-fd264b0fe244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.479408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0fbe1-9c77-4917-bc59-bface526ec99 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:43.496322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21198aed-683c-46bb-bacb-fd264b0fe244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:43.496322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21198aed-683c-46bb-bacb-fd264b0fe244 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:43.515271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:43.526959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:43.527629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.528067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.528352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.529839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=a76cd249767840b58a4b94babbbf4daf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:43.530161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:43.532088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39c1dc-e5d1-49dc-a1fa-e5b96a111632 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:43.540570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 131/266] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:43 2022] GET /identity/v3/role_assignments?user.id=a76cd249767840b58a4b94babbbf4daf&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:43.541962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fae2b1-897e-4a4a-8329-cacf916f6026 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.542251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fae2b1-897e-4a4a-8329-cacf916f6026 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.542533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fae2b1-897e-4a4a-8329-cacf916f6026 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.544168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 132/267] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:43.545373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461d97b-fec2-4ce1-895b-e3c5ab90a577 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.545659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461d97b-fec2-4ce1-895b-e3c5ab90a577 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.546034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461d97b-fec2-4ce1-895b-e3c5ab90a577 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.547537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 133/268] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:43.554023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.554331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.554591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.604630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:43.605826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:43.634960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 136/269] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:43.636421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.637072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.637364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.642285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:43.673812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:43.675588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:43.738671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 134/270] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:43.739535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:43.797504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 137/271] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:43.798451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:43.853579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:43.886611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd683c10-0cd0-4947-965d-32e45c55f96f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:43.932530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:43.933611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.933890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.934576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.936029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:43.937418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66402d45-8a55-43e8-80f5-7c168e3f1339 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:43.937531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:43.939251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 135/272] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:43 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:43.940019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:43.972870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:43.984765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:43.985576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:43.985858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:43.986134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:43.989403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:43.991064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:43.993204 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 138/273] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:43.994079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.068846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c88eb-a2c3-4331-bb15-8bb9e5b3e3db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.108285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b43f5b-cb76-4eea-9062-16057d270a00 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.113128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.114042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.114331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.114612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.122616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:44.128495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:44.138076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:44.138937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:44.139683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:44.140253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 136/274] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:44.144414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.157874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.158661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.158943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.159219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.160677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] RBAC: Adding query filter params (role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=a76cd249767840b58a4b94babbbf4daf) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:44.160992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:44.162674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:44.176575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 139/275] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:43 2022] GET /identity/v3/role_assignments?user.id=a76cd249767840b58a4b94babbbf4daf&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=c41b65addaec48bfb92f135a2a724132 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:44.177802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.267307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:44.312412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b5c3e-0596-43ea-a794-0721fe97cda0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.331724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9319c7-baef-4a52-90b0-47744e99e116 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.360828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124dfcaa-b84e-4d29-bdf4-f74fe26511e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.361656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dfcaa-b84e-4d29-bdf4-f74fe26511e0 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.362034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dfcaa-b84e-4d29-bdf4-f74fe26511e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.362405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124dfcaa-b84e-4d29-bdf4-f74fe26511e0 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.363266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dfcaa-b84e-4d29-bdf4-f74fe26511e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:44.365626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124dfcaa-b84e-4d29-bdf4-f74fe26511e0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:44.374856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.375995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.376229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.376658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.377539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:44.377720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:44.379306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:44.389850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 137/276] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:46:44 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:44.390599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ed2bb-9d8b-4881-bbb9-d39a8a4c85c6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.392164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ed2bb-9d8b-4881-bbb9-d39a8a4c85c6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.392164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ed2bb-9d8b-4881-bbb9-d39a8a4c85c6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.394392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 138/277] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:44.398602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 140/278] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:44.400378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.400969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.401453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.405535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.426417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:44.428029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:44.509582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:44.538218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 139/279] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:44.560696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.594797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b002507-3ba5-4441-a9ce-04f302f08116 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.636991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:44.647625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.648473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.649299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.649602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] PATH_INFO: `/v3/users/9d61ecbca02043a087cb517c69cbff56` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.667731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] RBAC: Authorizing `identity:get_user(user_id=9d61ecbca02043a087cb517c69cbff56)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:44.669343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:44.671105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 141/280] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:44 2022] GET /identity/v3/users/9d61ecbca02043a087cb517c69cbff56 => generated 293 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:44.677106 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.694073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0db24-5e0c-44c3-bb38-8f3228552b4d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.735766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.736940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.737465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.737965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.750097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:44.758841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:44.759927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:44.760930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:44.762991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 140/281] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:44.768793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:44.805071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:44.816297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168cad9-9919-4fd2-8944-7a8d2696d711 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.864987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.866482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.866974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.867882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.869288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, user.id=9d61ecbca02043a087cb517c69cbff56, scope.system=all) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:44.869812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:44.872569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660354a-a304-4a7a-afa6-a77590596543 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:44.880610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 142/282] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jul 15 13:46:44 2022] GET /identity/v3/role_assignments?user.id=9d61ecbca02043a087cb517c69cbff56&scope.system=all&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:44.897990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef0a3b-f58c-4413-aecc-054627754d15 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:44.913413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b636c-5e8c-4f48-80de-c70b5b27c8c6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.914025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b636c-5e8c-4f48-80de-c70b5b27c8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.914510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b636c-5e8c-4f48-80de-c70b5b27c8c6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.916479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 143/283] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:44.924551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:44.925979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.926359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.926715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.939608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:44.941254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:44.941714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:44.942160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:44.943108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:44.943593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:44.945263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:44.959343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 141/284] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:44.960894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:44.962362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:44.965039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.124157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967eb6e-dac3-4f10-b6ef-3b53c19950e2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.125759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 144/285] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:45.163795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.170181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.170181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.170181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.170181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] PATH_INFO: `/v3/users/ef993511e71041d59010fbe6b805617a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.175697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] RBAC: Authorizing `identity:get_user(user_id=ef993511e71041d59010fbe6b805617a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:45.177196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:45.179222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 142/286] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:44 2022] GET /identity/v3/users/ef993511e71041d59010fbe6b805617a => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:45.184537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.264584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:45.274771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:45.286245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d05c1-1c74-4bd7-8889-b8e154db5243 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.324608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.325428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.325714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.325993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.342072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:45.364977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:45.365891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:45.366656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:45.371496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 145/287] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:45.377527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983d4e8-8f16-4251-8ad1-967d59be54bc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.380005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.446452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.447241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.447523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.447897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.449197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:45.450650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:45.462036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 143/288] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:45 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:45.467988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.526870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90edd55-773d-476e-a5c8-1aeb04511bac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.564860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.565563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.565855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.566502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.567265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:45.567729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:45.569307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:45.578910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 146/289] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:45.585210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.604071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:45.697651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bde8c7-f687-4fc3-9845-fad61b5cfefc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.735998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.736899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.737181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.737646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] PATH_INFO: `/v3/users/94cc2e0008e74b79bd20287b43bdb1f1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.738852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] RBAC: Authorizing `identity:get_user(user_id=94cc2e0008e74b79bd20287b43bdb1f1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:45.740466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:45.742403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 147/290] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:45 2022] GET /identity/v3/users/94cc2e0008e74b79bd20287b43bdb1f1 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:45.747867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:45.806376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eeaa86-1570-4e26-a5d9-b6c4156ba5ae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.849055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:45.859960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:45.863356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f73451-56c2-46ec-8b3b-8e812a942712 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:45.903991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.904676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.904957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.905237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.933006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:45.933953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:45.934240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:45.934521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/ef993511e71041d59010fbe6b805617a/roles/625662ea5ad34314b41969de27beb3c5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:45.941314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=181d8587bb7b465483b60f3ec85eefda, user_id=ef993511e71041d59010fbe6b805617a, role_id=625662ea5ad34314b41969de27beb3c5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:45.942837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:45.976719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 144/291] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:46:45 2022] PUT /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda/users/ef993511e71041d59010fbe6b805617a/roles/625662ea5ad34314b41969de27beb3c5 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:45.986802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:45.988359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:46.078158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:46.100962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 148/292] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:46.106705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:46.271170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62d12d-6231-4aee-9d70-3fec7dd327a7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:46.318553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23904bb4-c4c6-4ca2-80dc-8e37cafb054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:46.319452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23904bb4-c4c6-4ca2-80dc-8e37cafb054e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:46.319733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23904bb4-c4c6-4ca2-80dc-8e37cafb054e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:46.320010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23904bb4-c4c6-4ca2-80dc-8e37cafb054e None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:46.321438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23904bb4-c4c6-4ca2-80dc-8e37cafb054e None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:46.322937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23904bb4-c4c6-4ca2-80dc-8e37cafb054e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:46.325131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 145/293] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:46 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:46.331377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:46.440245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e643f-1982-487e-8070-dfac30e46adf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:46.476608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:46.477261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:46.477544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:46.477833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:46.478651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=94cc2e0008e74b79bd20287b43bdb1f1) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:46.478957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:46.480474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07163b1-8a18-4583-849b-71e8aa7e7683 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:46.487224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 149/294] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:46 2022] GET /identity/v3/role_assignments?user.id=94cc2e0008e74b79bd20287b43bdb1f1&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:47.045485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1a67-6036-4d1b-9003-33349c86a38b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.045922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1a67-6036-4d1b-9003-33349c86a38b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.046024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1a67-6036-4d1b-9003-33349c86a38b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.047693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 146/295] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:47.053641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.054113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.054191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.071621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:47.072856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:47.111412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5611097-5353-4cb0-a0ad-9628457fd93a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.111727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5611097-5353-4cb0-a0ad-9628457fd93a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.111977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5611097-5353-4cb0-a0ad-9628457fd93a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.113488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 147/296] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:47.118637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.118926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.119212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.121321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.136280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:47.137812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:47.153364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 150/297] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:47.176278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.200841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.211013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 148/298] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:47.234333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.300077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74e45-152f-42cb-bbf5-f37b2a9d36cb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.340790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47110d9b-967e-4636-8846-6b2ea3a0a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.341956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47110d9b-967e-4636-8846-6b2ea3a0a561 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.342439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47110d9b-967e-4636-8846-6b2ea3a0a561 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.342913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47110d9b-967e-4636-8846-6b2ea3a0a561 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.343346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f26ac2-e71d-47e2-900f-9ed00cb1cd2f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.356378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47110d9b-967e-4636-8846-6b2ea3a0a561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-47110d9b-967e-4636-8846-6b2ea3a0a561 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:47.367593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:47.368554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:47.369352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:47.376759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 151/299] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:47.376759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878de3-215c-4f5b-bd35-2312b8944955 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.376759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878de3-215c-4f5b-bd35-2312b8944955 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.376759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55878de3-215c-4f5b-bd35-2312b8944955 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.376759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 152/300] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:47.380445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55878de3-215c-4f5b-bd35-2312b8944955 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.382747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45e5fb-3ad7-4db5-9eee-b714cb9f25be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.383622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e5fb-3ad7-4db5-9eee-b714cb9f25be None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.383909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e5fb-3ad7-4db5-9eee-b714cb9f25be None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.384185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45e5fb-3ad7-4db5-9eee-b714cb9f25be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.385519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45e5fb-3ad7-4db5-9eee-b714cb9f25be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:47.386998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45e5fb-3ad7-4db5-9eee-b714cb9f25be None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:47.389008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 149/301] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:47.395055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.395055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.395055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.407699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:47.408910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:47.422669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.454247 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.481090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 150/302] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:47.484578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.506327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55878de3-215c-4f5b-bd35-2312b8944955 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.542147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.543115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.543565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.544019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.544884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:47.545230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:47.546680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:47.560495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 153/303] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:47.560495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.595052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d783a30-bc3c-444b-bd82-c23aae47ad7a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.631917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.632617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.632901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.633181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] PATH_INFO: `/v3/groups` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.633901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:47.635363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:47.652733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 151/304] 10.222.0.16 () {60 vars in 1134 bytes} [Fri Jul 15 13:46:47 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:47.653484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.690095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.695398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ed18f-2df5-458d-94d7-a712b0a3ef43 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.702807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.746942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.748245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.748838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.749222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.766137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:47.778950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:47.780145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:47.780980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:47.781558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 154/305] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:47.787489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.794929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705058d2-5377-4b14-a867-2b8f03f82be9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.836326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.837638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.837921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.838193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] PATH_INFO: `/v3/users/c4251fcf38534bb885519064433b8174` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.853462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] RBAC: Authorizing `identity:get_user(user_id=c4251fcf38534bb885519064433b8174)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:47.854947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:47.856740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 152/306] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/users/c4251fcf38534bb885519064433b8174 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:47.862918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.911115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cc649-740b-4c50-8ff2-212139b9b271 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.950612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:47.952039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:47.952039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:47.952039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:47.952712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:47.953019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:47.954401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:47.963478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 155/307] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:47.968781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:47.979126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:47.986904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2a37e-a1f0-4887-b79b-37661cecacf6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:47.990169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:48.028072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:48.029159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.029717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.030306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.031357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] RBAC: Adding query filter params (role.id=625662ea5ad34314b41969de27beb3c5, user.id=c4251fcf38534bb885519064433b8174, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:48.031888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:48.033766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:48.039969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 153/308] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/role_assignments?user.id=c4251fcf38534bb885519064433b8174&scope.system=all&role.id=625662ea5ad34314b41969de27beb3c5 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:48.095236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc744e7-f804-4ce3-b021-5a5c54c65a34 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:48.137949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80d3e0-ffee-4f3f-806b-edf8cbebdd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:48.139290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80d3e0-ffee-4f3f-806b-edf8cbebdd6a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.139824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80d3e0-ffee-4f3f-806b-edf8cbebdd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.140302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80d3e0-ffee-4f3f-806b-edf8cbebdd6a None admin] PATH_INFO: `/v3/users/9d61ecbca02043a087cb517c69cbff56` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.141917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80d3e0-ffee-4f3f-806b-edf8cbebdd6a None admin] RBAC: Authorizing `identity:get_user(user_id=9d61ecbca02043a087cb517c69cbff56)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:48.143662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80d3e0-ffee-4f3f-806b-edf8cbebdd6a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:48.145731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 156/309] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:47 2022] GET /identity/v3/users/9d61ecbca02043a087cb517c69cbff56 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:48.151740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:48.270326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398b87c-6457-4f84-b652-f9147f7188db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:48.308641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7e2d3-30d4-4892-bc1d-19fe897b97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:48.309521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7e2d3-30d4-4892-bc1d-19fe897b97fd None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.309801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7e2d3-30d4-4892-bc1d-19fe897b97fd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.310078 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7e2d3-30d4-4892-bc1d-19fe897b97fd None admin] PATH_INFO: `/v3/system/users/9d61ecbca02043a087cb517c69cbff56/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.311573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7e2d3-30d4-4892-bc1d-19fe897b97fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9d61ecbca02043a087cb517c69cbff56, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:48.313021 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7e2d3-30d4-4892-bc1d-19fe897b97fd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:48.323613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 154/310] 10.222.0.16 () {58 vars in 1334 bytes} [Fri Jul 15 13:46:48 2022] PUT /identity/v3/system/users/9d61ecbca02043a087cb517c69cbff56/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:48.332854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84940e-8a99-4466-abe0-d0abd6ca9bc3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.333178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84940e-8a99-4466-abe0-d0abd6ca9bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.333426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84940e-8a99-4466-abe0-d0abd6ca9bc3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.334901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 157/311] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:48.341131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.341480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.341761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.361552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:48.363376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:48.382637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:48.447681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 155/312] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:48.476244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84940e-8a99-4466-abe0-d0abd6ca9bc3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:48.585739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:48.605392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84940e-8a99-4466-abe0-d0abd6ca9bc3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:48.642718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:48.643494 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.644356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.644658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.656257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:48.664967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:48.665872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:48.666628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:48.670910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 158/313] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:48.672693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:48.796604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56620a-e6a8-4629-911b-9800ab85ed35 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:48.834286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:48.834966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:48.835248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:48.835523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:48.836292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:48.836625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:48.838093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:48.848487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 156/314] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:48.857622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:48.971967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e86f39-c176-46ea-b360-e5cb2378648a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:49.008513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:49.009394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:49.009673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:49.009952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] PATH_INFO: `/v3/users/ef993511e71041d59010fbe6b805617a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:49.011187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] RBAC: Authorizing `identity:get_user(user_id=ef993511e71041d59010fbe6b805617a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:49.012673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:49.014326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 159/315] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:48 2022] GET /identity/v3/users/ef993511e71041d59010fbe6b805617a => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:49.020798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:49.039232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:49.131665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf890-3ddd-4bfa-abcf-f05f0e8be757 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:49.171852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:49.172539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:49.172821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:49.173096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:49.247339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:49.248770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:49.265434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:49.357183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 157/316] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:49.363148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:49.380346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:49.469831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad26aa6-1173-4523-9d0c-8e36268dee9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:49.507031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bd786-0270-434a-bb60-977514f5ee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:49.508341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bd786-0270-434a-bb60-977514f5ee01 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:49.508975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bd786-0270-434a-bb60-977514f5ee01 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:49.509509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bd786-0270-434a-bb60-977514f5ee01 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:49.510920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bd786-0270-434a-bb60-977514f5ee01 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:49.512637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bd786-0270-434a-bb60-977514f5ee01 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:49.519464 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 160/317] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:49 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:49.521095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:49.634286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efdf50-639b-4a7b-8a02-f79a6b6f02f8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:49.649797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:49.672793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:49.673471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:49.673761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:49.674041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:49.675441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] RBAC: Adding query filter params (role.id=625662ea5ad34314b41969de27beb3c5, scope.project.id=181d8587bb7b465483b60f3ec85eefda, user.id=ef993511e71041d59010fbe6b805617a) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:49.675779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:49.677366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad9cff-aea2-4c1d-a72e-e6411b4e8ad2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:49.684596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 158/318] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:46:49 2022] GET /identity/v3/role_assignments?user.id=ef993511e71041d59010fbe6b805617a&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=625662ea5ad34314b41969de27beb3c5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:49.892623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684492a0-2e8a-45dc-a4e2-f143e40f5a93 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:49.893011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684492a0-2e8a-45dc-a4e2-f143e40f5a93 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:49.893190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684492a0-2e8a-45dc-a4e2-f143e40f5a93 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:49.894802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 161/319] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:49.900351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:49.900697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:49.900960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:49.918978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:49.920234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:49.995265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 159/320] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:50.017204 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684492a0-2e8a-45dc-a4e2-f143e40f5a93 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.105295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:50.125874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684492a0-2e8a-45dc-a4e2-f143e40f5a93 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:50.171309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146dcc3-b590-40e4-a195-5aa4141192b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:50.172183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146dcc3-b590-40e4-a195-5aa4141192b3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.172497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146dcc3-b590-40e4-a195-5aa4141192b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.172777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146dcc3-b590-40e4-a195-5aa4141192b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.173964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146dcc3-b590-40e4-a195-5aa4141192b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:50.175474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146dcc3-b590-40e4-a195-5aa4141192b3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:50.177320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 162/321] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:46:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:50.182766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.245057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d8823-bd2a-496b-85a2-ac25be8e4303 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.245345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d8823-bd2a-496b-85a2-ac25be8e4303 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.245629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d8823-bd2a-496b-85a2-ac25be8e4303 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.247140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 163/322] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:50.251979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.252266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.252587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.269739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:50.270947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:50.288254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7e69f-f719-4db9-87d7-0fcf9642b386 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:50.298620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:50.329007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91397e18-9b73-47cb-be7b-4de7eb85a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:50.329689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397e18-9b73-47cb-be7b-4de7eb85a9c3 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.330134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397e18-9b73-47cb-be7b-4de7eb85a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.330421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397e18-9b73-47cb-be7b-4de7eb85a9c3 None admin] PATH_INFO: `/v3/groups` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.331659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91397e18-9b73-47cb-be7b-4de7eb85a9c3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:50.333147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91397e18-9b73-47cb-be7b-4de7eb85a9c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:50.345476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 164/323] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:50.351440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 160/324] 10.222.0.16 () {60 vars in 1134 bytes} [Fri Jul 15 13:46:50 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:46:50.368372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.461969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:50.486722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a393026-824d-4647-bdc4-fd7a7e99d7ce None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:50.538207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:50.539024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.539309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.539591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.542616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6feb0-86d7-4c35-9749-bc82d2e58fa5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.543118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6feb0-86d7-4c35-9749-bc82d2e58fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.543270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6feb0-86d7-4c35-9749-bc82d2e58fa5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.544621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 161/325] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:50.550239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.550523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.550794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.559906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:50.569050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:50.570175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:50.571513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 165/326] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:50.572347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:50.572814 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:50.576307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.676164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 162/327] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:46:50.703645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.729915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635b4f1-d460-4fad-94dc-3c1a1fc84b6b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:50.773735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:50.774676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.774963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.775243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.776013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:50.776322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:50.777761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:50.790063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 166/328] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:50.795657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.799059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:50.854372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:50.860064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54161fe-7bcc-45a2-8318-0a66899b6cbc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:50.907454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:50.908728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:50.909227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:50.909679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:50.923877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:50.934613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:50.935950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:50.937279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:50.942244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 163/329] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:46:50.951474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:50.975178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af213-d292-43dd-bba4-5718a1ab39fb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:51.020362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:51.021226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:51.021510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:51.021789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] PATH_INFO: `/v3/users/c4251fcf38534bb885519064433b8174` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:51.023076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] RBAC: Authorizing `identity:get_user(user_id=c4251fcf38534bb885519064433b8174)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:51.024614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:51.026712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 167/330] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:50 2022] GET /identity/v3/users/c4251fcf38534bb885519064433b8174 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:51.032259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:51.096530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846028d7-f022-4c77-a9e4-d16aa42f250d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:51.122730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:51.140801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:51.142032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:51.142789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:51.143490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:51.145277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:51.146052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:51.149048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:51.164668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1def3-8bce-4241-b9e3-606ee78c76a6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:51.172066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 164/331] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:51.178710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:51.208429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:51.208429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:51.208429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:51.208429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] PATH_INFO: `/v3/system/users/c4251fcf38534bb885519064433b8174/roles/625662ea5ad34314b41969de27beb3c5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:51.208888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c4251fcf38534bb885519064433b8174, role_id=625662ea5ad34314b41969de27beb3c5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:51.208888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:51.220726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 168/332] 10.222.0.16 () {58 vars in 1334 bytes} [Fri Jul 15 13:46:51 2022] PUT /identity/v3/system/users/c4251fcf38534bb885519064433b8174/roles/625662ea5ad34314b41969de27beb3c5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 13:46:51.224478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:51.355240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798a2e-e4d3-484e-bc0a-5aded35b47e3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:51.410426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e64b-5d92-4fa3-a9e7-82db39e0b661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:51.411347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e64b-5d92-4fa3-a9e7-82db39e0b661 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:51.411708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e64b-5d92-4fa3-a9e7-82db39e0b661 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:51.411986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e64b-5d92-4fa3-a9e7-82db39e0b661 None admin] PATH_INFO: `/v3/users/9d61ecbca02043a087cb517c69cbff56` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:51.413856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342e64b-5d92-4fa3-a9e7-82db39e0b661 None admin] RBAC: Authorizing `identity:get_user(user_id=9d61ecbca02043a087cb517c69cbff56)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:51.415505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342e64b-5d92-4fa3-a9e7-82db39e0b661 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:51.417893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 165/333] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:51 2022] GET /identity/v3/users/9d61ecbca02043a087cb517c69cbff56 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:51.424107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:51.578806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f36ab-792c-424c-9f0b-c321637bdd05 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:51.629196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:51.629853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:51.630144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:51.630434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:51.631294 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] RBAC: Adding query filter params (role.id=1773094ced584f8fbdccdb7ce4502fe4, user.id=9d61ecbca02043a087cb517c69cbff56, scope.system=all) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:51.631608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:51.633110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93767d8-e127-4370-ada2-063a5d7c82de None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:51.634904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:51.653556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 169/334] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jul 15 13:46:51 2022] GET /identity/v3/role_assignments?user.id=9d61ecbca02043a087cb517c69cbff56&scope.system=all&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:52.856234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39234-54dd-4fab-9963-b6281586b74b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:52.856823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39234-54dd-4fab-9963-b6281586b74b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:52.856823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39234-54dd-4fab-9963-b6281586b74b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:52.858757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 166/335] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:52.864346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:52.864877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:52.864979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:52.883774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:52.885029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:52.969458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 170/336] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:53.005853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39234-54dd-4fab-9963-b6281586b74b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:53.113463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:53.128914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39234-54dd-4fab-9963-b6281586b74b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:53.178995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:53.179820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:53.180106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:53.180409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:53.204838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:53.212775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:53.213753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:53.214515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 167/337] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:53.219976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:53.374107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73613fc2-66ce-42c4-8f85-986eb969660d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:53.409818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:53.423516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:53.424194 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:53.424508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:53.425302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:53.426074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:53.426382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:53.427810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28939ec-4e43-4241-8d51-2468cb296de9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:53.440580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 171/338] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:53.447973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:53.448858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587e02-315f-4c4c-8c20-432b87566a8e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:53.449149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587e02-315f-4c4c-8c20-432b87566a8e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:53.449432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587e02-315f-4c4c-8c20-432b87566a8e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:53.450882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 172/339] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:53.458804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:53.459165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:53.459401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:53.497345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:53.498540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:53.665383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d42a0-18a9-42eb-bcd2-68978ac2c837 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:53.684683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 173/340] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:53.726468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:53.740987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:53.741645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:53.741932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:53.742203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:53.773080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:53.879276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:53.881862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:53.934948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26268ee-66bd-41c5-88b7-5beb86dc6a47 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:53.957776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:53.993775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:53.995015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:53.995607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:53.996206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:54.003236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 168/341] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:54.014211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:54.022282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:54.025789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:54.026705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:54.027509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:54.030348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 174/342] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:54.053918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:54.082878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:54.141755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a294bb-8778-4aa6-91e0-c2b0588545ee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:54.178854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:54.179805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:54.180161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:54.180533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:54.181863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:54.183355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:54.188487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 169/343] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:54 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:54.192755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:54.255376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd099854-d4f1-4444-a98a-cc2dbc8541df None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:54.295904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:54.296610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:54.296893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:54.297168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:54.297925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:54.298223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:54.299629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:54.309506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 175/344] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:54.316472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:54.371692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:54.461004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad555931-44c4-4579-ad16-930a4ed74d3c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:54.489851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae115-4e72-4efe-8973-f9779b1aef9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:54.501075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:54.530602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:54.531476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:54.531852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:54.532538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:54.540675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:54.541057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:54.543832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 170/345] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:54 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:54.561696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:54.571205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:54.572259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:54.572774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:54.573267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:54.732142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:54.734571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:54.808167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce399c-eef5-4f87-ab38-de29cf80da08 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:54.854195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:54.891482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:54.892661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:54.893149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:54.894171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:54.895972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=f7e19f2b19b0472c87c1688310368276) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:54.896561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:54.899410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840991-353f-4897-8eff-de3eff28e96d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:54.921421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 171/346] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:46:54 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 260 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:55.023942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516754b7-e132-4f27-b7b7-42c6c3334ad9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.024667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516754b7-e132-4f27-b7b7-42c6c3334ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.025292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516754b7-e132-4f27-b7b7-42c6c3334ad9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.027691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 172/347] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:46:55.034216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ab44-1143-4245-9ce8-e44dc965cc74 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.035054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ab44-1143-4245-9ce8-e44dc965cc74 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.035785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ab44-1143-4245-9ce8-e44dc965cc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.056278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 176/348] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:55.078430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9be3ab44-1143-4245-9ce8-e44dc965cc74 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:55.080414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9be3ab44-1143-4245-9ce8-e44dc965cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:55.088191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:55.246765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:55.287821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 173/349] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:55.293696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156c9df-384f-45cd-a521-288ee4393fb5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.294300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156c9df-384f-45cd-a521-288ee4393fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.294808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156c9df-384f-45cd-a521-288ee4393fb5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.296525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 174/350] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:55.301104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac11384-dbfe-477c-bf5f-13931b75e866 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.301674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac11384-dbfe-477c-bf5f-13931b75e866 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.302156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac11384-dbfe-477c-bf5f-13931b75e866 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.304013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 175/351] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:55.305401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e050-ace2-43bc-8282-d0d1572d12ec None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.305913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e050-ace2-43bc-8282-d0d1572d12ec None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.306392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e050-ace2-43bc-8282-d0d1572d12ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.328704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5727e050-ace2-43bc-8282-d0d1572d12ec None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:55.330348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5727e050-ace2-43bc-8282-d0d1572d12ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:55.351740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef558a-3432-4b6f-afe8-7c7ea09b0e60 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:55.419472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:55.421043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.421690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.422198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.423560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 176/352] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:55.425252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.425773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.426248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.427061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:55.430372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:55.434223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 177/353] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:55.435949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:55.446231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:55.447690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:55.473010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:55.528663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 177/354] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:55.529841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:55.569905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:55.656592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba62511-a820-41ca-b15e-e1bbbd5b53da None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:55.694010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d5ff4-7a22-49a0-9438-3105131cad52 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:55.700989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:55.701862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.702138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.702406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.704435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:55.705930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:55.709136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 178/355] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:55.709968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:55.764071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:55.765762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.766447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.767129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.782742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:55.794591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:55.795621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:55.796470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:55.799028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 178/356] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:55.800150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:55.869423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ccc89-90e1-460f-b0a7-763532426f31 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:55.874982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:55.880794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:55.924137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64378b-54c0-4bfa-9687-54ce50d8e3db None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:55.942689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:55.944045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.944563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.945061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.965149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:55.965991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:55.966129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:55.966291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:55.966554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:55.980983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:55.981838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:55.982706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 13:46:55.983485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:55.984575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 179/357] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:55.985855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:55.989586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:55.990498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 179/358] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:55.992345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:56.124473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ad91c-10fa-4a38-83cd-c3f3d5e1f169 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:56.176824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:56.192158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:56.193155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:56.193599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:56.194039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:56.195050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:56.195656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:56.197127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:56.208126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6d21d-5407-4c4c-bc08-c90fc5e6f307 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:56.208420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 180/359] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:56.209092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:56.289021 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:56.290112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:56.290601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:56.291076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:56.292683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=f7e19f2b19b0472c87c1688310368276) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:56.293209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:56.295961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:56.312564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 180/360] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:46:55 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=c41b65addaec48bfb92f135a2a724132 => generated 260 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:56.314120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:56.364621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:56.436496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fb732-67f1-49f5-8333-9a926b1541d3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:56.462831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de562865-9f02-4476-b360-ebbcbabf9e97 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:56.505747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:56.506679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:56.507106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:56.507686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:56.508696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:56.509181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:56.510804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:56.520870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 181/361] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:56.522130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:56.578751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:56.579892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:56.580418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:56.580907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:56.582367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:56.582974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:56.585511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:56.610533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 181/362] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:56.614062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:56.630706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:56.645917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe75f8-8716-4775-918f-8617ba3e95d5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:56.698272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e61624-3099-4805-9066-9301d280a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:56.699768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e61624-3099-4805-9066-9301d280a743 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:56.700554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e61624-3099-4805-9066-9301d280a743 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:56.701064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e61624-3099-4805-9066-9301d280a743 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:56.792261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e61624-3099-4805-9066-9301d280a743 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:56.794082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e61624-3099-4805-9066-9301d280a743 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:56.809471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:56.895206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:56.910877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 182/363] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:56.911740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e61624-3099-4805-9066-9301d280a743 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:56.913804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81bfc9-5d64-40b1-8a28-4fad1119e81e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:56.996255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:56.997701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:56.998479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:56.999163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.044465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e61624-3099-4805-9066-9301d280a743 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:57.094854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473a9c3-16dd-4d3f-aafd-ea4c772779bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:57.095853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473a9c3-16dd-4d3f-aafd-ea4c772779bd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.096295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473a9c3-16dd-4d3f-aafd-ea4c772779bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.096629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473a9c3-16dd-4d3f-aafd-ea4c772779bd None admin] PATH_INFO: `/v3/users/c4251fcf38534bb885519064433b8174` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.101235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473a9c3-16dd-4d3f-aafd-ea4c772779bd None admin] RBAC: Authorizing `identity:get_user(user_id=c4251fcf38534bb885519064433b8174)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:57.102775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473a9c3-16dd-4d3f-aafd-ea4c772779bd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:57.107131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 183/364] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:46:56 2022] GET /identity/v3/users/c4251fcf38534bb885519064433b8174 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:57.108495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6b1d4-2c7e-4ca1-ba0f-177bda3a00e3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.108782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6b1d4-2c7e-4ca1-ba0f-177bda3a00e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.109060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6b1d4-2c7e-4ca1-ba0f-177bda3a00e3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.110375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 184/365] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:57.111146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6b1d4-2c7e-4ca1-ba0f-177bda3a00e3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:57.118706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:57.120551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:57.223031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:57.229847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:57.301428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 182/366] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:57.301987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:57.330223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6b1d4-2c7e-4ca1-ba0f-177bda3a00e3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:57.407324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e43c-417c-46bd-8afa-e5eae2e02ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:57.409937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e43c-417c-46bd-8afa-e5eae2e02ced None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.410857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e43c-417c-46bd-8afa-e5eae2e02ced None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.411711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e43c-417c-46bd-8afa-e5eae2e02ced None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.414212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e43c-417c-46bd-8afa-e5eae2e02ced None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:57.417714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e43c-417c-46bd-8afa-e5eae2e02ced None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:57.422103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 185/367] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:57 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:57.424102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.429116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.429846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.454760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:57.459261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:57.480431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f0b18-1620-4cee-a4a5-bec8a04d09ae None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:57.529401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:57.530350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.530641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.530923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.532269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] RBAC: Adding query filter params (role.id=625662ea5ad34314b41969de27beb3c5, user.id=c4251fcf38534bb885519064433b8174, scope.system=all) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:57.532632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:57.534767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:57.559872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 186/369] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:57.561096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:57.561201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 183/369] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jul 15 13:46:57 2022] GET /identity/v3/role_assignments?user.id=c4251fcf38534bb885519064433b8174&scope.system=all&role.id=625662ea5ad34314b41969de27beb3c5 => generated 510 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:57.563099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:57.596053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:57.655836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:57.720754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e216c-b5a1-4fbd-a79d-745e49e6e705 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:57.775967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:57.776971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.777700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.778015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.781963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:57.783618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:57.786752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 187/370] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:57 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:57.787425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:57.822805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801de7-d5e4-4df8-a38d-f0204a4679b2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:57.874121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee96298-2a8e-4567-94aa-b4f96fbd7361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:57.875157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96298-2a8e-4567-94aa-b4f96fbd7361 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.875510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96298-2a8e-4567-94aa-b4f96fbd7361 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.875939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96298-2a8e-4567-94aa-b4f96fbd7361 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.877260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee96298-2a8e-4567-94aa-b4f96fbd7361 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:57.879608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee96298-2a8e-4567-94aa-b4f96fbd7361 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:57.880861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 184/371] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:57 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:57.884427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67e3f-07f7-43bd-a54d-945a8b4e7975 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:57.884427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67e3f-07f7-43bd-a54d-945a8b4e7975 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:57.884427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67e3f-07f7-43bd-a54d-945a8b4e7975 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:57.884427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 185/372] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:57.885279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc67e3f-07f7-43bd-a54d-945a8b4e7975 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:57.947519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fe110-bcae-4f0f-9315-7442541c003d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:57.983430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:57.995729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:58.013074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.014239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.014628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.014993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.034632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:58.037930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc67e3f-07f7-43bd-a54d-945a8b4e7975 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:58.043028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:58.044153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:46:58.045538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:58.052092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 188/373] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:58.052092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.108848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58bede-88bc-4f1e-8131-b38a41f73dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.110264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58bede-88bc-4f1e-8131-b38a41f73dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.110723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58bede-88bc-4f1e-8131-b38a41f73dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.111193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58bede-88bc-4f1e-8131-b38a41f73dc3 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.114180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58bede-88bc-4f1e-8131-b38a41f73dc3 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:58.116803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58bede-88bc-4f1e-8131-b38a41f73dc3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:58.120216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 186/374] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:57 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:58.124509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.124509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.124509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.144958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:58.146229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:58.207367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562d98e-06b9-409a-a3b5-639d282265eb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.257438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 187/375] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:58.259147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.260126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.261383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.261883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.262376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.264168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=181d8587bb7b465483b60f3ec85eefda) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:58.264594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:58.269602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:58.278971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 189/376] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:58.280602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.356092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:58.376536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:58.395444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf54b04-b627-4356-b52c-3dd72b64febb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.438610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.439529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.439820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.440108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.442087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:58.442417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:58.445546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:58.460065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 188/377] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:58.460065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.472236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d66a15-5eec-4f28-a525-2fbc77691350 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.523578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.524738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.525263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.525772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.526888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=181d8587bb7b465483b60f3ec85eefda) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:58.527448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:58.530100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:58.537897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=c41b65addaec48bfb92f135a2a724132 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:58.539367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.615071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16530e8a-a7af-43f5-8c80-7460826b621d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.688281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.689492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.690003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.690542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.695162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:58.715987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:58.726713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:58.727648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:46:58.728430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:46:58.736367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 189/379] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:58.743571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafb59c-abe1-4659-b55d-71b13a499f72 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.744937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.768488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:58.836996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.838044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.838529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.839000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.888116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fbf51-8f2e-4c5d-acc1-232adba21681 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:58.944160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:58.944853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:58.945139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:58.945418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:58.946187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:58.946493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:58.948081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:58.958113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 190/380] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:58.965647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:58.967085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:58.968610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.085245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:59.095711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 191/381] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:46:59.102705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.136793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:59.143231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18771-07d9-49c6-b19e-cea72b064f6c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.182121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.183002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.183292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.183572 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.184822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.186483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.188639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 191/382] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:58 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:59.197826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.259913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93849aa1-0901-4ef5-8fce-fa8a62833a4f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.297316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfab412-e936-48a5-b1e1-7f41974fc440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.298463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfab412-e936-48a5-b1e1-7f41974fc440 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.298886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfab412-e936-48a5-b1e1-7f41974fc440 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.299298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfab412-e936-48a5-b1e1-7f41974fc440 None admin] PATH_INFO: `/v3/projects/8618b94bef7144e799c6d51bb1ef841f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.300769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfab412-e936-48a5-b1e1-7f41974fc440 None admin] RBAC: Authorizing `identity:get_project(project_id=8618b94bef7144e799c6d51bb1ef841f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.302967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfab412-e936-48a5-b1e1-7f41974fc440 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.305235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 192/383] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/projects/8618b94bef7144e799c6d51bb1ef841f => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:59.306740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633fb0-0834-4441-9009-3c5771b08039 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.307164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633fb0-0834-4441-9009-3c5771b08039 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.307591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633fb0-0834-4441-9009-3c5771b08039 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.309065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 193/384] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:46:59.314339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633fb0-0834-4441-9009-3c5771b08039 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.320582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac2c05-25c9-4bac-b437-54793e2bf029 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.352202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:59.366692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb70629-88ae-4e1e-9c11-48ae0ea85450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.367859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb70629-88ae-4e1e-9c11-48ae0ea85450 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.368289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb70629-88ae-4e1e-9c11-48ae0ea85450 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.368760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb70629-88ae-4e1e-9c11-48ae0ea85450 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.370350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb70629-88ae-4e1e-9c11-48ae0ea85450 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.372191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb70629-88ae-4e1e-9c11-48ae0ea85450 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.374601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 192/385] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:59.376111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.376696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.377195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.397231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:46:59.398831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:46:59.432189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:59.475562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633fb0-0834-4441-9009-3c5771b08039 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.497095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 193/386] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:46:59.498349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.523670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.526430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.527818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.531185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] PATH_INFO: `/v3/groups/6283340384eb4beab07c6b5314149c42` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.539438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] RBAC: Authorizing `identity:get_group(group_id=6283340384eb4beab07c6b5314149c42)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.541404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.544743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 194/387] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/groups/6283340384eb4beab07c6b5314149c42 => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:46:59.545953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.641828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41eabcb-d1ac-47ac-a589-cd95dfcc9285 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.667253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407ea8d-90c5-4472-a39a-25a47bfb9814 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.675074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:59.693567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.695024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.695024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.695024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.698158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e19f2b19b0472c87c1688310368276, group_id=bc48ecf79cdd4443a4fd41b320dc28b1, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.700019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.718849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 194/388] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jul 15 13:46:59 2022] PUT /identity/v3/projects/f7e19f2b19b0472c87c1688310368276/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:46:59.719435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.719787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.720997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.721493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.722059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.735945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:46:59.745610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:46:59.746547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:46:59.747418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:46:59.756075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 195/389] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:46:59.758074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:46:59.762350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:46:59.870361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad25291-e248-40c5-b6fc-af9b82e93d57 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.911087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943223f-8c1c-4a45-8f7f-2c30fe29c7d8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:46:59.919729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.920831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.921348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.921882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.922973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] RBAC: Adding query filter params (group.id=6283340384eb4beab07c6b5314149c42, role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=8618b94bef7144e799c6d51bb1ef841f) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:59.923499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.925801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.934666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 195/390] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/role_assignments?group.id=6283340384eb4beab07c6b5314149c42&scope.project.id=8618b94bef7144e799c6d51bb1ef841f&role.id=10895c39249448e4ac3616747cafa76b => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:59.955874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:46:59.956695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:46:59.956980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:46:59.957258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:46:59.958066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:46:59.958378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:46:59.959879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:46:59.970600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 196/391] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:46:59.980367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:00.067180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:00.150042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eb1e8-5809-428d-9b60-f15dc5c6b505 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:00.201775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:00.202651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:00.202929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:00.203203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:00.209248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:00.210738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:00.212829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 196/392] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:46:59 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:00.219185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:00.344580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:00.435451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69061da-2a02-48d5-9ff2-204706c264e5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:00.520752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd27c9-0c09-45ff-9c7e-41d639d0e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:00.522795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd27c9-0c09-45ff-9c7e-41d639d0e148 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:00.523611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd27c9-0c09-45ff-9c7e-41d639d0e148 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:00.524448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd27c9-0c09-45ff-9c7e-41d639d0e148 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:00.527384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd27c9-0c09-45ff-9c7e-41d639d0e148 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:00.536429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd27c9-0c09-45ff-9c7e-41d639d0e148 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:00.541140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 197/393] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:00 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:00.551620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:00.691178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f31594-2565-4b1f-af77-ea85ac366ad7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:00.713482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:00.740839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8cf6f-c8ec-4c6c-8f43-b7420cc75bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:00.742151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8cf6f-c8ec-4c6c-8f43-b7420cc75bcf None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:00.742631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8cf6f-c8ec-4c6c-8f43-b7420cc75bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:00.743129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8cf6f-c8ec-4c6c-8f43-b7420cc75bcf None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:00.745921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8cf6f-c8ec-4c6c-8f43-b7420cc75bcf None admin] RBAC: Authorizing `identity:create_grant(project_id=f7e19f2b19b0472c87c1688310368276, group_id=bc48ecf79cdd4443a4fd41b320dc28b1, role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:00.747626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8cf6f-c8ec-4c6c-8f43-b7420cc75bcf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:00.766582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 197/394] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jul 15 13:47:00 2022] PUT /identity/v3/projects/f7e19f2b19b0472c87c1688310368276/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/c41b65addaec48bfb92f135a2a724132 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:01.114443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9ac8-7d8e-43cc-b985-372030a0af05 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.114937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9ac8-7d8e-43cc-b985-372030a0af05 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.115246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9ac8-7d8e-43cc-b985-372030a0af05 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.118183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 198/395] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:01.123600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.123894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.124180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.144483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:01.145800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:01.237274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 198/396] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:01.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9ac8-7d8e-43cc-b985-372030a0af05 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:01.430616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9ac8-7d8e-43cc-b985-372030a0af05 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:01.482986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:01.494072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:01.495263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.495742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.496231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.509902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:01.518134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:01.519062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:47:01.519805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:01.525003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 199/397] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:01.528238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:01.667243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57aaae-79c5-4c85-b85e-0eb6b0cbefbb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:01.673448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:01.720515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:01.721135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.721414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.722220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.722973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:01.723276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:01.724995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:01.734160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 199/398] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:47:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:01.740190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:01.869174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787f84-bb93-4d2b-b15d-2b4ed684a817 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:01.911403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab085f-5c33-491e-aab1-9a1f41c8d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:01.912346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab085f-5c33-491e-aab1-9a1f41c8d91b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.912655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab085f-5c33-491e-aab1-9a1f41c8d91b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.912928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab085f-5c33-491e-aab1-9a1f41c8d91b None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.914098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab085f-5c33-491e-aab1-9a1f41c8d91b None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:01.915599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab085f-5c33-491e-aab1-9a1f41c8d91b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:01.917599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 200/399] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:01 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:01.926065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:01.940063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfd314-3974-4d19-af54-f6d569aac879 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.940429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfd314-3974-4d19-af54-f6d569aac879 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.940687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfd314-3974-4d19-af54-f6d569aac879 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.944579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 201/400] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:01.952424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:01.952424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:01.952424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:01.972364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:01.974584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:02.050979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:02.066807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 202/401] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:47:02.092612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.181801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2c176-21ae-4556-8134-63c94019b956 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.211484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83d8b-dc3d-42d1-acf1-8328abcb99b3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.252551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.254157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.254645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.255121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.267297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:02.275783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.276689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.276972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.277418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:02.278359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:02.279225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:02.280003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:02.280632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:02.280632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:02.281966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 200/402] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:01 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:02.288690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 203/403] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:47:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:02.289274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.294209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.355148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:02.379274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:02.421459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ff1d2-b03a-4232-ac25-f98785b2c2d7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.465497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.466822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.467337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.468033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.469980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:02.470516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:02.472211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cccb2-026e-4f53-ac6f-713303277ad3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:02.485676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 204/404] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:02.491802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9921-45ae-4e73-a1d9-4f4fc6f5b350 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.492528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9921-45ae-4e73-a1d9-4f4fc6f5b350 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.492701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e9921-45ae-4e73-a1d9-4f4fc6f5b350 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.494295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 205/405] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:02.494890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9921-45ae-4e73-a1d9-4f4fc6f5b350 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.510832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e6138-260f-4446-b7e3-0233c9a442f2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.547760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f73283-2efb-412d-92b3-5bb9dbe5c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.548787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73283-2efb-412d-92b3-5bb9dbe5c68f None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.549069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73283-2efb-412d-92b3-5bb9dbe5c68f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.549388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73283-2efb-412d-92b3-5bb9dbe5c68f None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.553944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73283-2efb-412d-92b3-5bb9dbe5c68f None admin] RBAC: Authorizing `identity:create_grant(project_id=181d8587bb7b465483b60f3ec85eefda, group_id=bc48ecf79cdd4443a4fd41b320dc28b1, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:02.555774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73283-2efb-412d-92b3-5bb9dbe5c68f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:02.575787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 201/406] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jul 15 13:47:02 2022] PUT /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:02.577665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6d65-7f9e-4764-9754-dc57ff47de6a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.577965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6d65-7f9e-4764-9754-dc57ff47de6a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.578247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6d65-7f9e-4764-9754-dc57ff47de6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.601519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b09f6d65-7f9e-4764-9754-dc57ff47de6a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:02.603265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b09f6d65-7f9e-4764-9754-dc57ff47de6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:02.651192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e9921-45ae-4e73-a1d9-4f4fc6f5b350 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.668115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:02.693929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.695194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.695688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.696330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.698082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:02.699844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:02.701916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 206/408] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:02 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:02.707002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 202/408] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:02.707731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c952de1-61ec-4bc0-a77a-7ed50f7e2445 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.708034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c952de1-61ec-4bc0-a77a-7ed50f7e2445 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.708314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c952de1-61ec-4bc0-a77a-7ed50f7e2445 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.709332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.710109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 203/409] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:02.716144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.716490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.716735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.743551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:02.745226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:02.764779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:02.821924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 204/410] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:02.822918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.830627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce63eef-9d71-4f6a-bf08-65831bfac897 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.871716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.872613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.872891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.873165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:02.874391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:02.876452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:02.879287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 207/411] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:02 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:02.883908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:02.948829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:02.949401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c2c2-d3d2-4a43-8c9d-d96e183d2f21 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:02.988337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:02.989549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:02.990053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:02.990528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.002331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:03.011328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:03.012201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:03.013047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:03.020515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc398d0-a8e2-479d-b08f-5adedfa51f0a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.020886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 205/412] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:03.021969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.039682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:03.062805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.063607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.063889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.064162 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.076663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:03.085030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:03.085980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:03.087185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 208/413] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:03.088036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.160452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b88993-2675-43fe-bd6f-456a6f918e52 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.201197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5597e0-1fec-488b-baae-2be669661e65 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.206880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.208326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.208752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.209032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.211830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] RBAC: Authorizing `identity:create_grant(project_id=181d8587bb7b465483b60f3ec85eefda, group_id=bc48ecf79cdd4443a4fd41b320dc28b1, role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:03.213340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:03.213919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:03.231964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 206/414] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jul 15 13:47:03 2022] PUT /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda/groups/bc48ecf79cdd4443a4fd41b320dc28b1/roles/c41b65addaec48bfb92f135a2a724132 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:03.233378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.262863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.264529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.266010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.266548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.267900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:03.269780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:03.272582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:03.293682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 209/415] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:03.298746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.405094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:03.457916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc78b7-02b9-422d-9ff6-8b32b8a24f91 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.484960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772c39c-fff3-49af-ac82-5402b0243b49 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.506018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.506704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.506989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.507268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.508734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:03.508734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:03.510354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934a03d-5599-464e-ab18-600c447d1360 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:03.529219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 207/416] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:47:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:03.529547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4a089-75c2-4ac6-be0d-9bf53ea9de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.530005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4a089-75c2-4ac6-be0d-9bf53ea9de76 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.530285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4a089-75c2-4ac6-be0d-9bf53ea9de76 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.530465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8fe1d-8a3f-4d16-abcc-4ef30cd59e05 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.530555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4a089-75c2-4ac6-be0d-9bf53ea9de76 None admin] PATH_INFO: `/v3/groups/6283340384eb4beab07c6b5314149c42` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.530750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8fe1d-8a3f-4d16-abcc-4ef30cd59e05 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.531036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8fe1d-8a3f-4d16-abcc-4ef30cd59e05 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.531802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4a089-75c2-4ac6-be0d-9bf53ea9de76 None admin] RBAC: Authorizing `identity:get_group(group_id=6283340384eb4beab07c6b5314149c42)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:03.532281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 208/417] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:03.533383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4a089-75c2-4ac6-be0d-9bf53ea9de76 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:03.538630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8fe1d-8a3f-4d16-abcc-4ef30cd59e05 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.539908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 210/418] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:03 2022] GET /identity/v3/groups/6283340384eb4beab07c6b5314149c42 => generated 229 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:03.541338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.541627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.541911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.562926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:03.564640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:03.606311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:03.655737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 211/419] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:03.657161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.667094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8fe1d-8a3f-4d16-abcc-4ef30cd59e05 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.723135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.724990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.725422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.726103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.744895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:03.788811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78288b-fc53-48c8-8963-14686e613483 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:03.834325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:03.837074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:03.840189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:03.841490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:03.842830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:03.843311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] PATH_INFO: `/v3/projects/8618b94bef7144e799c6d51bb1ef841f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:03.844794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=8618b94bef7144e799c6d51bb1ef841f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:03.846448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:03.849301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 212/420] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:03 2022] GET /identity/v3/projects/8618b94bef7144e799c6d51bb1ef841f => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:03.850296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:03.907232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:03.985636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 209/421] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:03.986795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:04.010864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2647-d493-42ca-9547-84e1efbc7bf6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.031111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:04.063908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.064760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.065040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.066266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.087103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:04.097735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:04.099668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 213/422] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:47:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:04.101158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:04.128338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad937b-3fba-4489-bc8c-a93bf58db089 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.183272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.184233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.184563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.184840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] PATH_INFO: `/v3/projects/8618b94bef7144e799c6d51bb1ef841f/groups/6283340384eb4beab07c6b5314149c42/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.187133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] RBAC: Authorizing `identity:create_grant(project_id=8618b94bef7144e799c6d51bb1ef841f, group_id=6283340384eb4beab07c6b5314149c42, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:04.189531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:04.210035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 210/423] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jul 15 13:47:03 2022] PUT /identity/v3/projects/8618b94bef7144e799c6d51bb1ef841f/groups/6283340384eb4beab07c6b5314149c42/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:04.211304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:04.228468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:04.246116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03374f6-2647-485b-b320-ded67f227da2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.324024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.324908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.325199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.325549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.329453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:04.331006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:04.333377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 214/424] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:04 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:04.337644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:04.338641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:04.344980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f729ff0-5a32-45cc-9eb0-77349dd13c5d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.395617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.396333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.396641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.396917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.397698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:04.398001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:04.399420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:04.425172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 211/425] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:04.435015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:04.525790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd906-51a5-4306-a4bb-ed4a95aa6226 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.570795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.571512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.571910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.572356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.573880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:04.575538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:04.577770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 215/426] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:04 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:04.583757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:04.591078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70cbb4-602b-41b6-9b76-fc05d3e77a75 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.624752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:04.658148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041ce64-34e1-4923-a493-3e3f996f372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.659335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041ce64-34e1-4923-a493-3e3f996f372e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.659844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041ce64-34e1-4923-a493-3e3f996f372e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.660337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041ce64-34e1-4923-a493-3e3f996f372e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.727068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:04.775738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777ffb-5349-4928-a873-749a3450a440 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:04.778036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041ce64-34e1-4923-a493-3e3f996f372e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:04.779538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041ce64-34e1-4923-a493-3e3f996f372e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:04.840085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:04.840708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:04.841002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:04.841292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:04.842162 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=f7e19f2b19b0472c87c1688310368276) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:04.842485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:04.843967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:04.859217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 216/427] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:47:04 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 611 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:04.893089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 212/428] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:04.898677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:05.038654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:05.048912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5520f-7e2c-437b-aca0-69ac2f407bc8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:05.090465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc700c70-4b6d-4291-af0a-45a56c86c8d4 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.091115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc700c70-4b6d-4291-af0a-45a56c86c8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.091602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc700c70-4b6d-4291-af0a-45a56c86c8d4 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.093672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 213/429] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:47:05.100561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.101173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.101650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.104342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:05.105255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.105541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.105822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] PATH_INFO: `/v3/projects/f7e19f2b19b0472c87c1688310368276` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.107198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] RBAC: Authorizing `identity:get_project(project_id=f7e19f2b19b0472c87c1688310368276)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:05.108718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:05.110819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 217/430] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:04 2022] GET /identity/v3/projects/f7e19f2b19b0472c87c1688310368276 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:05.119451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:05.129731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:05.130962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:05.234038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:05.256922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ca286-ae89-4225-bdaa-924cd491a92b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:05.262549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 214/431] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:47:05.295926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:05.296838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.297124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.297408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.298684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:05.300239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:05.302392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 218/432] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:05 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:05.311880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:05.325359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:05.380185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:05.523091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e43bfc-3607-4ed5-b4b8-f2931175bbdd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:05.549040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dc5c-03ff-474d-8b7c-f8f74fa74c78 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:05.588774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:05.589453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.589728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.590002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.590857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=f7e19f2b19b0472c87c1688310368276) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:05.591168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:05.595878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e55f8-a7ef-43fe-ab77-080491ee0a50 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:05.601827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 215/433] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:47:05 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=f7e19f2b19b0472c87c1688310368276&role.id=c41b65addaec48bfb92f135a2a724132 => generated 611 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:05.602390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119b121-771a-4470-a5da-e2b77064b8c1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.602680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119b121-771a-4470-a5da-e2b77064b8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.603987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119b121-771a-4470-a5da-e2b77064b8c1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.608791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 216/434] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:05.610293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040354a3-3599-4525-a196-273227235eba None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.610582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040354a3-3599-4525-a196-273227235eba None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.610857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040354a3-3599-4525-a196-273227235eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.630318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-040354a3-3599-4525-a196-273227235eba None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:05.631454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-040354a3-3599-4525-a196-273227235eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:05.633293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:05.635359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.636170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.636932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.660619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:05.679057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:05.680711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 13:47:05.682258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:05.687329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 219/435] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:05.689054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:05.698082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:05.713223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 217/436] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:05.738744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040354a3-3599-4525-a196-273227235eba None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:05.902484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:05.916249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e9ccb-4e77-4326-b863-4e488f5a01c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:05.926591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040354a3-3599-4525-a196-273227235eba None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:05.967509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:05.969142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.970045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.970905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.973219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:05.974372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:05.977855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:05.979855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:05.980805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:05.981085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:05.981365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:05.995414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:05.996939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 220/437] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:47:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:06.004374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:06.006155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:06.007718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:06.011119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 218/438] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:47:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:06.012629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:06.013375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:06.074904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:06.174748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b09b3b-d6ac-4ff3-a1b0-e56a3868ce94 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:06.200009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071c4e-cf1b-4a9d-b3b7-a8e997224291 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:06.240947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:06.241798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.242076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.242353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.245383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:06.247090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.247871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.248667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.250364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:06.251244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:06.255312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:06.271423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:06.277722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 219/439] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:06.283333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:06.344212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:06.345842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:06.464305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1a882-9467-4d32-aa28-005f22a5b14a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:06.469466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:06.489017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 221/440] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:06.497580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:06.533139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20a6a-3a63-446b-be42-465051e3cfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:06.534139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20a6a-3a63-446b-be42-465051e3cfbd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.534529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20a6a-3a63-446b-be42-465051e3cfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.534988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20a6a-3a63-446b-be42-465051e3cfbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.592554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:06.651022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20a6a-3a63-446b-be42-465051e3cfbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:06.653762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20a6a-3a63-446b-be42-465051e3cfbd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:06.672906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dcb36-ac31-4c89-9afb-401b9c7a4f45 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:06.718301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa2c12-8681-4440-a1d7-a219bcdf08eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:06.719247 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa2c12-8681-4440-a1d7-a219bcdf08eb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.719626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa2c12-8681-4440-a1d7-a219bcdf08eb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.719983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa2c12-8681-4440-a1d7-a219bcdf08eb None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.721364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa2c12-8681-4440-a1d7-a219bcdf08eb None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:06.722929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa2c12-8681-4440-a1d7-a219bcdf08eb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:06.725167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 222/441] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:06.727326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1e7b6-848c-43b6-a9f1-04c0a30a616c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.727812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1e7b6-848c-43b6-a9f1-04c0a30a616c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.728292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1e7b6-848c-43b6-a9f1-04c0a30a616c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.731042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 223/442] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:06.734786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1e7b6-848c-43b6-a9f1-04c0a30a616c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:06.814226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 220/443] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:06.818005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.818510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.819008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.827878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:06.844812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:06.846753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:06.868595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1e7b6-848c-43b6-a9f1-04c0a30a616c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:06.902267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:06.918570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:06.919604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:06.919994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:06.920411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:06.921835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:06.923402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:06.925796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 224/444] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:06.926639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:06.944208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 221/445] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:06.945660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:07.075468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda643-3c7e-49ce-887d-6ce7234229be None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.087806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eadf35-4615-42b9-90ad-c72b35ec9f9b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.123243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.124239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.124661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.125004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] PATH_INFO: `/v3/projects/181d8587bb7b465483b60f3ec85eefda` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:07.126384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] RBAC: Authorizing `identity:get_project(project_id=181d8587bb7b465483b60f3ec85eefda)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:07.128143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:07.130114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 225/446] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/projects/181d8587bb7b465483b60f3ec85eefda => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:07.131384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:07.135206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.136020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.136517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.137280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:07.138329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=1773094ced584f8fbdccdb7ce4502fe4, scope.project.id=181d8587bb7b465483b60f3ec85eefda) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:07.138857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:07.140458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:07.142708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:07.150102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 222/447] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:47:06 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=1773094ced584f8fbdccdb7ce4502fe4 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:07.151751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:07.238401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:07.320065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3341b-0c19-4e80-8e3c-93ea0c5b6ed6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.329198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfbe53-923a-408c-8299-d169c4d9d3bf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.375167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.377053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.377569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.378060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] PATH_INFO: `/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:07.380203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] RBAC: Authorizing `identity:get_group(group_id=bc48ecf79cdd4443a4fd41b320dc28b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:07.383063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:07.386267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 223/448] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:07 2022] GET /identity/v3/groups/bc48ecf79cdd4443a4fd41b320dc28b1 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:07.393154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:07.394412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.395752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.396311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.396836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:07.416331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:07.432023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:07.433180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:07.434155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:07.434776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 226/449] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:07.441200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:07.529647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:07.583296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9376db-e841-46de-9d34-a2469e0a0be4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.625162 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bd2b9-0ff0-4f1b-9474-c26fe1cf2e44 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.633808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.634495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.634782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.635062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:07.635928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] RBAC: Adding query filter params (group.id=bc48ecf79cdd4443a4fd41b320dc28b1, role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=181d8587bb7b465483b60f3ec85eefda) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:07.636244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:07.637777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:07.645512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 224/450] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:47:07 2022] GET /identity/v3/role_assignments?group.id=bc48ecf79cdd4443a4fd41b320dc28b1&scope.project.id=181d8587bb7b465483b60f3ec85eefda&role.id=c41b65addaec48bfb92f135a2a724132 => generated 611 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:07.660666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:07.694716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.695853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.696356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.696890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:07.698399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:07.698932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:07.701768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:07.724459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 227/451] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:07.726208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:07.848795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fb9f9-07e3-4061-b910-0a7121c56b9f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:07.889916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:07.898043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:07.898913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:07.899386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:07.899850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:08.002469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:08.004649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:08.130322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 225/452] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:08.136245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:08.249168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92c67a-517a-4920-acbf-9bec9317a124 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:08.308606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:08.309961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:08.310529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:08.310967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] PATH_INFO: `/v3/projects/8618b94bef7144e799c6d51bb1ef841f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:08.312947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=8618b94bef7144e799c6d51bb1ef841f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:08.316158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:08.320464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 228/453] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:47:08 2022] GET /identity/v3/projects/8618b94bef7144e799c6d51bb1ef841f => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:08.326588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:08.359349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:08.437583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9441e-168f-4f8d-a7b9-19d163404989 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:08.473872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e72102-5275-45cd-8876-eb576109aea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:08.475003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72102-5275-45cd-8876-eb576109aea0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:08.475287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72102-5275-45cd-8876-eb576109aea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:08.475566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e72102-5275-45cd-8876-eb576109aea0 None admin] PATH_INFO: `/v3/groups/6283340384eb4beab07c6b5314149c42` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:08.476783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e72102-5275-45cd-8876-eb576109aea0 None admin] RBAC: Authorizing `identity:get_group(group_id=6283340384eb4beab07c6b5314149c42)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:08.478231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e72102-5275-45cd-8876-eb576109aea0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:08.479862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 226/454] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jul 15 13:47:08 2022] GET /identity/v3/groups/6283340384eb4beab07c6b5314149c42 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:08.486159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:08.539024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:08.591715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be457f-69bd-4645-a74b-c42d049ad3a1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:08.626249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:08.626896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:08.627176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:08.627449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:08.628852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] RBAC: Adding query filter params (group.id=6283340384eb4beab07c6b5314149c42, role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=8618b94bef7144e799c6d51bb1ef841f) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:08.629160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:08.630607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc13dc9-12e5-4ea7-b75d-86aa353e7c6d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:08.636795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 229/455] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jul 15 13:47:08 2022] GET /identity/v3/role_assignments?group.id=6283340384eb4beab07c6b5314149c42&scope.project.id=8618b94bef7144e799c6d51bb1ef841f&role.id=10895c39249448e4ac3616747cafa76b => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:11.144659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbea2c-009c-4d37-83af-7b4c9d65c5fb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.145739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbea2c-009c-4d37-83af-7b4c9d65c5fb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.146244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbea2c-009c-4d37-83af-7b4c9d65c5fb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.148218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 227/456] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:11.154918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6bf48-0bbc-4c13-a509-da8975683d87 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.155421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6bf48-0bbc-4c13-a509-da8975683d87 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.155895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6bf48-0bbc-4c13-a509-da8975683d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.196308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-25e6bf48-0bbc-4c13-a509-da8975683d87 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:11.198444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-25e6bf48-0bbc-4c13-a509-da8975683d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:11.372340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 230/457] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 13:47:11.416511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbea2c-009c-4d37-83af-7b4c9d65c5fb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:11.580261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:11.586301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbea2c-009c-4d37-83af-7b4c9d65c5fb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:11.668636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:11.670437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.671207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.672006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.696286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:11.699363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:11.721046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574a3e2-8ac1-4fdb-9288-c1f54549a331 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.722237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574a3e2-8ac1-4fdb-9288-c1f54549a331 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.722750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574a3e2-8ac1-4fdb-9288-c1f54549a331 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.728568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 231/458] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:11.730206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db139f39-ae19-4227-b0f8-5e4185305077 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.730702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db139f39-ae19-4227-b0f8-5e4185305077 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.731179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db139f39-ae19-4227-b0f8-5e4185305077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:11.741057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:11.741967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:11.742887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:11.742887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:11.742887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:11.742887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:11.743745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 228/459] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:11.754886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:11.778240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-db139f39-ae19-4227-b0f8-5e4185305077 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:11.780577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db139f39-ae19-4227-b0f8-5e4185305077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:11.842269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:11.883095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 232/460] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:11.885110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c13641-c247-46ab-ac4a-d7c5175d1963 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.885557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c13641-c247-46ab-ac4a-d7c5175d1963 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.886013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c13641-c247-46ab-ac4a-d7c5175d1963 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.887778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 233/461] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:11.894958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:11.895477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:11.895921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:11.914855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:11.916312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:11.978811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f882b-559a-4905-9b5a-12f5af524c2b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:11.997703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 234/462] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:11.998960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.050510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:12.051535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.051965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.052788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.053731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:12.054893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:12.057003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:12.074618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 229/463] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:12.075893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.100980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:12.191692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b430c-4672-4107-a1b4-371c8b6925cc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:12.194372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:12.246756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:12.247930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.248457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.248965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.262481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:12.264326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:12.285855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555faa9-0585-45d8-8f07-761e50a68823 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:12.287688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:12.288550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:12.289378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:12.289378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:12.289378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:12.289378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:12.296584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 235/464] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:12.297908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.377643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:12.378575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.378876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.379223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.393727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:12.395679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:12.417342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:12.418943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:12.420589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:12.420589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:12.420589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:12.420589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:12.420589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 230/465] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:12.421758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.433041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b40b2-2052-4dca-8486-6396abb403bf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:12.479789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6a930-df83-4182-9ba8-423460fa4cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:12.481289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6a930-df83-4182-9ba8-423460fa4cff None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.481981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6a930-df83-4182-9ba8-423460fa4cff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.482367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6a930-df83-4182-9ba8-423460fa4cff None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.483603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6a930-df83-4182-9ba8-423460fa4cff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:12.486177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6a930-df83-4182-9ba8-423460fa4cff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:12.516523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 236/466] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:47:12 2022] POST /identity/v3/users => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:12.517957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:12.518978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03870a71-dcd7-4147-92f8-8e38b943c5f7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.519359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03870a71-dcd7-4147-92f8-8e38b943c5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.519743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03870a71-dcd7-4147-92f8-8e38b943c5f7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.522379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 237/467] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:12.523588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03870a71-dcd7-4147-92f8-8e38b943c5f7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.546764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:12.614230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc0972-ea5b-416a-b354-7a045a475dd6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:12.680159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:12.681458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.681917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.682375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.683758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:12.684273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:12.686782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790be8e-5658-4f5f-870a-438876db003d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:12.709997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 231/468] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:12.711754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.712250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.712847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.733154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:12.734814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:12.745618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03870a71-dcd7-4147-92f8-8e38b943c5f7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:12.808769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:12.810088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:12.810670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:12.811252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:12.812867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:12.813527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:12.816212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:12.825030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 232/469] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:12.826082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.841626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 238/470] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:12.848498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:12.905795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:12.946063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:12.959748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f6da-c9b2-4e08-9864-28ad8cf9eec8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:12.981964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89e1d-036a-4057-9731-e9d760d5ff29 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:13.001274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.001959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.002240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.003054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.004229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:13.006045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:13.018946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8d455-c1e0-440c-a77f-ce1033c99541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.019659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8d455-c1e0-440c-a77f-ce1033c99541 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.019913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8d455-c1e0-440c-a77f-ce1033c99541 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.020244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8d455-c1e0-440c-a77f-ce1033c99541 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.021055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8d455-c1e0-440c-a77f-ce1033c99541 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:13.022668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8d455-c1e0-440c-a77f-ce1033c99541 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:13.031548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 233/471] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:47:12 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:13.033506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:13.056852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 239/472] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:47:12 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:47:13.060098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfef27-1539-4291-90b0-a123eac1b395 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.061089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfef27-1539-4291-90b0-a123eac1b395 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.061658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfef27-1539-4291-90b0-a123eac1b395 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.064790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 240/473] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:13.074409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.074764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.078779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.121182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:13.122477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:13.240668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822bcc-bd2e-4f0d-bc56-b48acf48ebc4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:13.264322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:13.306710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 241/474] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:13.326346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.327666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.328167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.328730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.329945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:13.350045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:13.351617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:13.379720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:13.381482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:13.383149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:13.383149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:13.383149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:13.383149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:13.383547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 234/475] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:13.389870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:13.438083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:13.537099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe976-706b-4f90-ab6a-8f8284833e73 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:13.603842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.604463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.604750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.605027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.605806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:13.607211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:13.617198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa64a-55ef-45fb-b0e5-eae5b85d5fde None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:13.626070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 13:47:13.626070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 13:47:13.626070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [parameters: {'id': '8620acef40cb4e348c48522915e8c05f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 13:47:13.626070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 15 13:47:13.651195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return ref.to_dict() Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 13:47:13.652952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 13:47:13.654815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 13:47:13.657876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 13:47:13.661418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 13:47:13.665488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'id': '8620acef40cb4e348c48522915e8c05f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:13.669546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 13:47:13.673210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:13.676965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 242/476] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jul 15 13:47:13 2022] POST /identity/v3/roles => generated 144 bytes in 323 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:13.676965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:13.709450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.710187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.710479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.710761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.711557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:13.711877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:13.713452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:13.749163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 235/477] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:13.757337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:13.804962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:13.853002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea9f93-004a-45ee-bf13-30109b682c6b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:13.900027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:13.949908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e681e8e-dee3-4b80-b1b0-fba7a5445ea9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:13.951003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.952526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.953044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.953546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.988339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:13.994037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17a649-6ff4-4e58-8a5e-381fb627e80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:13.994992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a649-6ff4-4e58-8a5e-381fb627e80a None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:13.995271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a649-6ff4-4e58-8a5e-381fb627e80a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:13.995543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a649-6ff4-4e58-8a5e-381fb627e80a None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:13.996271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17a649-6ff4-4e58-8a5e-381fb627e80a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:14.003640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:14.004691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 13:47:14.005444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:14.006515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17a649-6ff4-4e58-8a5e-381fb627e80a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:14.013115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 243/478] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:47:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:14.014081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:14.044341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 236/479] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:47:13 2022] POST /identity/v3/users => generated 249 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:14.191031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ffe08-52b4-479c-884f-2b218aa59379 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:14.238238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:14.238930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:14.239212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:14.239489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:14.240283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:14.240626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:14.242044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148a0f8-a022-41f4-a827-25454dbce28a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:14.251482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 244/480] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:15.487612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978ec7a-66ca-41a9-b620-d4c1e474bc9a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:15.488639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978ec7a-66ca-41a9-b620-d4c1e474bc9a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:15.489141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978ec7a-66ca-41a9-b620-d4c1e474bc9a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:15.491134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 237/481] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:15.498666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7b8ab-fa1c-4434-accc-152f3a9bbd0f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:15.499611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7b8ab-fa1c-4434-accc-152f3a9bbd0f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:15.500096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7b8ab-fa1c-4434-accc-152f3a9bbd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:15.526194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-49a7b8ab-fa1c-4434-accc-152f3a9bbd0f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:15.527792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-49a7b8ab-fa1c-4434-accc-152f3a9bbd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:15.582320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:15.629882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 245/482] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:15.697491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978ec7a-66ca-41a9-b620-d4c1e474bc9a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:15.790189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:15.812433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19417d68-c6dd-4ca7-9650-8f1d55469bc7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:15.812709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19417d68-c6dd-4ca7-9650-8f1d55469bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:15.813029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19417d68-c6dd-4ca7-9650-8f1d55469bc7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:15.814620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 246/483] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:15.823401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:15.823621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978ec7a-66ca-41a9-b620-d4c1e474bc9a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:15.824487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:15.825244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:15.865533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:15.866371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:15.866656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:15.866938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:15.880373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:15.889280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:15.890879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:15.892349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 238/484] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:15.895065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:15.898309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:15.907741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:16.031765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142f01d-0ef3-4eb9-9c14-d1035f1e12e4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:16.073199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:16.073896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:16.074185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:16.074470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:16.075255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:16.075566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:16.077042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:16.087965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 239/485] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:16.088296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:16.103880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:16.208085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 247/486] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:16.250122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b366af-8376-461f-ba1d-094b51fc7a84 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:16.275371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:16.292627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:16.293817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:16.294307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:16.294785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:16.308964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:16.310856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:16.332092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:16.332980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:16.333783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:16.333783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:16.333783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:16.333783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:16.336095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 240/487] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:16.343783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:16.408635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:16.473328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b5801-fa0e-4a36-b9d0-35b15d988bd0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:16.534556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:16.535730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:16.536236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:16.536765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:16.538227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:16.538800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:16.541461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:16.592226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 241/488] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:16.599304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:16.651665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601d6d2-cf6b-4015-8fbf-9067eeb21508 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:16.654443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:16.728835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:16.730158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:16.730700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:16.731149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:16.752425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:16.774253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:16.775284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:16.776052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:16.776705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 248/489] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 502 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:16.787066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:16.918408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfea15-b947-44d0-939d-9579188a10e4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:16.929264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960ccfd-7461-4211-a9d2-7f24a81a42a8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:16.943083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:16.973599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:16.974552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:16.974838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:16.975122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:16.975904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:16.976215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:16.978342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95bcb-6854-4da0-89d0-bf4aaf7a6572 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:16.990339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 249/490] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:16.991801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e336fb-1b8f-4245-b2ae-2b8723de6c95 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:16.992099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e336fb-1b8f-4245-b2ae-2b8723de6c95 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:16.992382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e336fb-1b8f-4245-b2ae-2b8723de6c95 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.000050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 250/491] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:17.000050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e336fb-1b8f-4245-b2ae-2b8723de6c95 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:17.004345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:17.006362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.007160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.007941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.024045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:17.025572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:17.029506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1642b279-aab3-453b-b2f6-eab46fa1d0e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:17.066224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:17.068073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:17.069892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:17.081604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 242/492] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:47:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:17.084495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.085344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.086276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.117529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e336fb-1b8f-4245-b2ae-2b8723de6c95 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:17.121576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:17.124070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:17.156848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:17.157658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.157959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.158290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.170586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:17.172013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:17.192311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:17.193943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:17.195637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:17.195637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:17.195637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:17.195637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:17.195637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 251/493] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:17.195637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:17.280434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 243/494] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:17.280434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:17.308849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:17.359988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2baa2e-a47d-4cad-b451-9747cb7d6e08 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:17.410161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:17.411264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:17.411863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.412142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.412465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.413311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:17.413650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:17.415105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4830bee-70af-41ac-a862-6fc7917392aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:17.415732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076e90-5ee6-4efe-89a5-7babbcef5205 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:17.467719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 252/495] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:47:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:17.467719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9286e8-72f6-4109-8693-c7659920dfe0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.467719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9286e8-72f6-4109-8693-c7659920dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.467719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9286e8-72f6-4109-8693-c7659920dfe0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.471403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 253/496] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:17.472706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9286e8-72f6-4109-8693-c7659920dfe0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:17.483065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:17.483954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.484359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.484828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.485755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:17.486192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:17.487999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:17.504201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 244/497] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:17.505782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:17.653661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9286e8-72f6-4109-8693-c7659920dfe0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:17.654440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0a35-4a8f-41a8-9efb-031cdcf38629 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:17.711732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e50d6-d9a9-4b15-81c2-89a8f5cf888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:17.712962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e50d6-d9a9-4b15-81c2-89a8f5cf888e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.713480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e50d6-d9a9-4b15-81c2-89a8f5cf888e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.713995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e50d6-d9a9-4b15-81c2-89a8f5cf888e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.715230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e8bd2-949c-4116-9fd2-6e33c3063f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:17.716234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e8bd2-949c-4116-9fd2-6e33c3063f6f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.716686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e8bd2-949c-4116-9fd2-6e33c3063f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.717081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e8bd2-949c-4116-9fd2-6e33c3063f6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.720052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:17.733438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e8bd2-949c-4116-9fd2-6e33c3063f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b46e8bd2-949c-4116-9fd2-6e33c3063f6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:17.742698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:17.744976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 254/498] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:17.748302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.751681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.751681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.766495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:17.768613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:17.770929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:17.830391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e50d6-d9a9-4b15-81c2-89a8f5cf888e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:17.832751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e50d6-d9a9-4b15-81c2-89a8f5cf888e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:17.867907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 255/499] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:17.869716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:17.968774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 245/500] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:17.971216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983825fd-e014-4b5e-82e8-946cc4b5521d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:17.971576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983825fd-e014-4b5e-82e8-946cc4b5521d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:17.971986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983825fd-e014-4b5e-82e8-946cc4b5521d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:17.973532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 246/501] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:17.974639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983825fd-e014-4b5e-82e8-946cc4b5521d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.022559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605d92c-7f29-439d-ba34-41c8bce43374 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.052420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:18.056087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:18.068702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.069595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.070052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.070419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.079882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.081850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:18.101701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:18.103706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 256/502] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:18.105698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.115411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983825fd-e014-4b5e-82e8-946cc4b5521d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.162487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.163648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.164004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.164341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.165566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:18.166026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.168482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.182542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 247/503] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:18.183911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.258195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cf00c-79d9-499f-b659-44bf0663d934 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.312209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.313456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.313863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.314250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.328344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.330382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.333893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4753c10-7b61-41f7-92c3-f84c3992aff5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c11a88a5-313c-47e7-9c9d-51fe4b490798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:18.352092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:18.353785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:18.355108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:18.355108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:18.355108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:18.355108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:18.355108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 257/504] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:18.355804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.356221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.356717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.369951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:18.380568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:18.382461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:18.389340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.390983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.391445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.391905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.410285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.412900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.431621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:18.443489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:18.444754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:18.445780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:18.445780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:18.445780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:18.445780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:18.445780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 248/505] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:18.446827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.508550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 258/506] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:18.508550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.595176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69817a3f-88c9-441d-a4a3-ba3c6b843be6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.633350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fb554-d2b0-4bd1-9dce-fb2a9ace8298 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.653889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.655053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.655605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.656037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.657698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:18.658219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.661421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.670759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.671563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.671845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.672121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.683233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.684752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:18.703319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:18.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:18.707744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:18.707744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:18.707744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:18.707744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:18.710315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 259/507] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:18.711248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.742040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 249/508] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:18.742040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.742523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:18.784164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:18.834134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c642c4-f610-4716-a3aa-f3c5bcc5b20d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.877320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.878148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.878500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.878859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.879729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:18.880108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.881728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.897794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b72264-4c83-4a30-a23c-6c9ae8eb81b1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:18.898926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 260/509] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:18.899861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:18.951913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:18.952851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:18.953303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:18.953693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:18.954509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:18.954861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:18.957206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:18.971624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 250/510] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:18.973100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.001596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:19.030832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dc6cf-9271-486b-88b3-a1919772e722 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:19.084239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:19.085216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:19.085547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:19.085866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:19.100122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:19.102274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:19.110726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:19.112738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 261/511] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:19.114318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.123379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5a40-a5e0-4bd2-901a-dd28f67b9e49 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:19.169638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:19.170458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:19.170873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:19.171245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:19.172090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:19.172533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:19.174201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:19.194117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 251/512] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:19.195424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.261398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbfc93-28ce-49ff-adee-7c2d577c23c9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:19.312689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:19.313555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:19.313998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:19.314459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:19.330388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:19.360744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab610b1-8cb5-4c46-855b-1f716e35833c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:19.416811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:19.417772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:19.418170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:19.418618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:19.419570 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:19.421178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:19.426945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:19.428668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:19.448377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 252/513] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:47:19 2022] POST /identity/v3/users => generated 247 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:19.450060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.467457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:19.625633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 262/514] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:19.627908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.633675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7faa7c8-717b-472a-bd3c-73b173bbe50d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:19.691694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:19.701429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:19.702554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:19.703015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:19.703475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:19.718243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:19.721454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:19.751726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:19.753708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:19.754949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 253/515] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:19.755524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.811175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af560cda-88db-49d1-8d58-60b5ebd0d202 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:19.870424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:19.871421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:19.871901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:19.872298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:19.873510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:19.874019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:19.876495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:19.885599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:19.893449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 263/516] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:19.894495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:19.947551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae765e-9fd6-43d2-b624-54871922df8a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.009834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.011128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.011591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.012056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.025622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.028167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:20.054830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:20.056639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 254/517] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:20.059483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.083095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:20.088939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abafc7a-3b32-4dc8-9488-fad94fda6dc6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.157657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.162797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.162797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.162797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.178112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.180572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:20.211787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:20.214030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:20.214030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:20.214030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:20.214030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:20.214030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 264/518] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:20.214030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.238622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5a14-f6ab-4103-b6b5-63692dad758b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.280902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:20.292914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.293600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.293882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.294190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.295486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:20.295797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.297636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.324659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 255/519] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:20.326819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.397671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d40ed-c67a-496a-9511-b41dc8f4f205 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.454685 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.456032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.456314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.456646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.469363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.470980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:20.490624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:20.500061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:20.502547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:20.502547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:20.502547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:20.502547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:20.502547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 265/520] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:20.502547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.506387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:20.511702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aabf3e-c8e5-47dd-a5ae-9ec5009f241c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.563736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.564446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.565265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.565545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.566395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:20.566701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.568157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.594077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 256/521] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:20.594915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.652040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:20.672225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c7841-9e31-4865-8ec5-8f1c9cc76fc6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.716371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.717319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.717319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.717560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.718244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:20.718546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.719966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.720344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82183f2b-6e5a-4702-845c-cbbf1e0ce50c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.735537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 266/522] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:20.736524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.770022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.770715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.770996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.771270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.772132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=e424ea6d03414857bc408e49b8df5429) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:20.772478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.773965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:20.780713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 257/523] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/role_assignments?user.id=e424ea6d03414857bc408e49b8df5429&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:20.781592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:20.845932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:20.929753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ca52d-40e0-4ada-8b78-43a6fa8cb561 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:20.979646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:20.981166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:20.981842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:20.982485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:20.984013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:20.984798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:20.988052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:21.000119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1735ed-1f3f-4f00-b18b-e4a063994db9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:21.012466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 267/524] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:21.014264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:21.023685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:21.068467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:21.069490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:21.069964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:21.070430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:21.210737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:21.214495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:21.226610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389795ca-078b-4e11-99e6-33c2b66877f6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:21.277095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:21.302971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:21.304866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:21.305922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:21.306655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:21.323019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:21.327172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:21.364238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:21.365641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:21.366550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:21.369855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 268/525] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:21.372064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:21.423953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 258/526] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:21.426280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:21.452098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:21.575492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d15f44-7dc1-4b33-818e-740c8023005a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:21.622488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:21.623883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:21.624347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:21.625393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:21.635264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:21.636744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941805c-df0b-4ddc-be99-ff50e01cf282 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:21.637244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:21.656686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:21.657881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 13:47:21.659668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:21.662714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 269/527] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:21.664142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:21.708024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:21.709331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:21.709912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:21.710461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:21.712105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:21.712817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:21.715687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:21.722952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:21.748812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 259/528] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:21.750567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:21.790057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db8d01-79cf-4eec-9839-080542b77d8a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:21.831234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:21.832301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:21.832760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:21.833206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:21.854414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:21.856153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:21.876813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:21.877993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:21.878871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:21.878871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:21.878871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:21.878871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:21.880481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 270/529] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:21.881478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:21.905157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:21.974064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab151eca-c894-4690-abbf-75ad151035a2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:21.999609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:22.007845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1ca43-5957-4a81-b6e3-6fabff58180d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.039309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.040550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.041063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.041543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.043168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:22.043727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.046587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be963098-7804-4bd0-8775-00ea249360cc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.050743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.051418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.051726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.051999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.052896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=deb87accdcab4759add4b76958d151a8) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:22.053217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.054734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.061088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 271/530] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:21 2022] GET /identity/v3/role_assignments?user.id=deb87accdcab4759add4b76958d151a8&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:22.061940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.092495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 260/531] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Jul 15 13:47:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:22.095530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8b27-8ea2-411a-8181-b32c34c323ff None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.096497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8b27-8ea2-411a-8181-b32c34c323ff None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.096990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf8b27-8ea2-411a-8181-b32c34c323ff None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.102142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 261/532] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:22.103723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8b27-8ea2-411a-8181-b32c34c323ff None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.210479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1654f1-5e8c-4103-beff-cf8915783f0f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.263647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.264704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.265748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.266227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.267490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:22.268642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.271233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a1e3-9e0f-4495-8b32-e6101d02616c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.283695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf8b27-8ea2-411a-8181-b32c34c323ff None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.288030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 272/533] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:22.289565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.289930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.290282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.310073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:22.311546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:22.319919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:22.330639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.331454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.331817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.332171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.334607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:22.410925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 273/534] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:22.412738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.414250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.415859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.547279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 262/535] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:22.548062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.570208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec578d39-5beb-4560-b4f2-70114deb32ae None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.591085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:22.610849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.611958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.612308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.612695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.620802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.622458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:22.638735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:22.639635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:22.640482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 274/536] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:22.641721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.657301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:22.703305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4be34f-4a13-477f-bcff-87b4cfc3bd3d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.748059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.749005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.749556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.749556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.762330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:22.770710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:22.771646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:22.772448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:22.776371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762f084-d769-4a1f-8e93-35148fb55f8e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.777072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 263/537] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:22.778332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.827354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.829173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.829667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.830186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.847159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.849908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:22.874762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:22.875820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:22.876952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:22.876952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:22.876952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:22.876952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:22.879054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 275/538] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:22.880366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.889592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:22.918551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73df3f-427b-46fe-bd9e-57d372a78b71 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:22.962427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:22.963189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:22.963819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:22.963819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:22.964853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:22.965264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:22.966880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:22.982137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 264/539] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:22.983152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:22.984973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:23.077257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bf551-6395-4825-8854-7713a1739ac2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.113033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe94a53-f0aa-4b90-8727-ae62e74ea762 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.136263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.137471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.137751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.138190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.139082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:23.139383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.141455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.155750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 276/540] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:23.156790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.166659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.166659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.166659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.166659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.167285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:23.167285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.167285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dc767-120e-4bbf-acc0-c799bb9d9b28 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.188131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 265/541] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:23.188131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a02a-098f-41ba-ab87-217bdec919e0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.188131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a02a-098f-41ba-ab87-217bdec919e0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.188131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a02a-098f-41ba-ab87-217bdec919e0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.188131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 266/542] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:23.188131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967a02a-098f-41ba-ab87-217bdec919e0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.202286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:23.298560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c6aad-c0cc-4d88-80c6-7c35c849505f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.324740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:23.338169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.338714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.338996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.339270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.340574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=f0e94eecf28f42468e14bbd0c4eb3777) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:23.340574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.341930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.356425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 277/543] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/role_assignments?user.id=f0e94eecf28f42468e14bbd0c4eb3777&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:23.356425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.362810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967a02a-098f-41ba-ab87-217bdec919e0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.445698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.447586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.448173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.448806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.473322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.476248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-91c6e8ca-141a-4381-bec9-950460c987ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:23.514851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:23.515746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:23.516597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:23.516597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:23.516597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:23.516597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:23.516800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abf19c-afd0-433e-bde0-2f27a184cde1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.517319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 267/544] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:23.520531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.521108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.521619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.554225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:23.556316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:23.557735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.558549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.558888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.559168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.570663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.571658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:23.595781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:23.596728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:23.597564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:23.600467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 278/545] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:23.601570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.646901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:23.665382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 268/546] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:23.667280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.669379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:23.766937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c388dd-4683-4afb-ba93-a4d55adbcd9b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.812254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccaa6d-9e40-4620-b161-247c9cfde3d5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:23.835011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.836298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.837320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.837676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.839133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:23.839821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.842977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.853982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:23.854883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:23.855300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:23.855707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:23.856672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:23.857139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:23.858761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:23.869295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 279/547] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:23.871012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.876801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 269/548] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:23.878304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:23.946580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:24.015108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdf7d6-b60f-4644-992d-f82a0f90194f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:24.087327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:24.088026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.088296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.088592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.095083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a93f1d-2115-484e-a2ec-bf620603cf53 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:24.107700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:24.112278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:24.118234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:24.149994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:24.150828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.151106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.151374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:24.160632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:24.163118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:24.165039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 270/549] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:24.166084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:24.172880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:24.181734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:24.183446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:24.185056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:24.188376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 280/550] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:24.189441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:24.347999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72914952-58ee-46c4-9182-322d762bb1a2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:24.362010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:24.390553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:24.391256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.391526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.391800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.392237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae27d6-0bd1-4710-aa64-8507d4ddde56 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:24.392852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=393c4bea16074727958ea40ab497948d) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:24.393162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:24.394738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:24.404511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 271/551] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:24 2022] GET /identity/v3/role_assignments?user.id=393c4bea16074727958ea40ab497948d&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:24.406411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:24.478441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:24.479129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.479400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.479679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.480562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:24.480870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:24.482454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:24.524451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 281/552] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jul 15 13:47:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:24.530075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:24.602817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:24.661274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff0f7f-0456-40c7-ada7-ec72ae018390 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:24.736362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:24.738054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.738849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.739631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.739800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1ab20-b11a-49c6-aeef-962332e88294 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:24.741522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:24.742407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:24.745353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb6c4-e284-47af-89b5-8d0a8c6579c9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:24.764861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 272/553] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:24.768245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0a24b-4ab7-40ef-9ddb-07e761a9e59b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.769067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0a24b-4ab7-40ef-9ddb-07e761a9e59b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.769865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0a24b-4ab7-40ef-9ddb-07e761a9e59b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.773263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 273/554] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:24.775019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0a24b-4ab7-40ef-9ddb-07e761a9e59b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:24.786936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb220b8-2ac3-4844-9875-e02f12c014b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:24.787611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb220b8-2ac3-4844-9875-e02f12c014b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:24.787893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb220b8-2ac3-4844-9875-e02f12c014b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:24.788167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb220b8-2ac3-4844-9875-e02f12c014b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:24.847615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:24.866875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb220b8-2ac3-4844-9875-e02f12c014b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:24.868252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb220b8-2ac3-4844-9875-e02f12c014b6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:24.899750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:24.977992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0a24b-4ab7-40ef-9ddb-07e761a9e59b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.001408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 282/555] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:25.003443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.003701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.003984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.019900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.021136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.021649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.022173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.022903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:25.024231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:25.035712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:25.037635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:25.057120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:25.058289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:25.059305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:25.059305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:25.059305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:25.059305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:25.061232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 274/556] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:25.062227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.159442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 283/557] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:25.161296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.165482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:25.182600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfd3da-dc05-40ef-b7de-16e3e7f679ee None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.221965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.222971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.223362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.223998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.237898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:25.239611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:25.259960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:25.262596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:25.264902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:25.264902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:25.264902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:25.264902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:25.264902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 275/558] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:25.264902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.293543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:25.382544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba6f6-8f3e-4e6b-91c4-66dcb68501ee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.392935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f18cf8-c99d-4c40-bd93-1b29eec3467d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.434373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.435457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.435740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.436025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.446189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:25.450205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:25.456486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.457759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.458318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.458765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.459771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:25.460236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:25.461827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:25.469924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:25.471019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:25.471920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:25.474171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 276/559] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:25.475553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.484509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 284/560] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:25.485745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.598084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666b443-a57f-40f2-ad9b-240b3bf99b73 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.617770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:25.639514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.640493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.640932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.641348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.642432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:25.642967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:25.644581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:25.654521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 285/561] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:25.655806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.696027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014db179-98d1-43e5-a985-a7ac4003fc0f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.785461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218af62-5906-4fa7-b686-6e9e17fffe38 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:25.791492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.792732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.793241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.793746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.796219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:25.796811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:25.802009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:25.830449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:25.831257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:25.831540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:25.831817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:25.833702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 277/562] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:25.835341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.849035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:25.850510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:25.870227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:25.871657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:25.872590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 286/563] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:25.873664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:25.917669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:25.934130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:25.971682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a74-07bf-4322-8242-60ac94888976 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.022113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.022500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03c9d1-6d03-436e-893c-1550d56f9f9d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.022927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.023213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.023491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.035664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.037182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.060944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.061736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.062014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:26.062198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:26.064061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:26.066760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:26.066760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:26.066760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:26.066760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:26.067254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.067404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 278/564] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:26.067404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.075532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.076831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:26.091777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:26.093968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:26.096117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:26.100644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 287/565] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:26.100644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.183077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494c9ab-8a90-4c1b-89ae-b3361effb931 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.199068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:26.216677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:26.220862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff7f12-0218-41b7-ab78-72da02fa172e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.229414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.230399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.230836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.231262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.232449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:26.232922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.234700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.257925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 279/566] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:26.259368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.263641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.265375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.265663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.266318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.267187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:26.267522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.269012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.292594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 288/567] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:26.293637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.386766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee7b3-e3b9-41f8-b1e9-e79e91dc7501 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.410733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f719fd-db35-4d96-b69e-e6fc6c8d4b68 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.430354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.431530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.432028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.432557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.433573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:26.434209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.435902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.449994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.450800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.451079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.451352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.452188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 280/568] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:26.453365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.468504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.469771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:26.491214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:26.492123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:26.493529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:26.493529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:26.493529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:26.493529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:26.493529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 289/569] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:26.493999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.494918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:26.507993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:26.584573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560c161-9489-4cb6-b44a-728b53c3f5e9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.625968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494f363-9903-4f3f-b347-da524712bc56 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.635306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.636342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.636801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.637073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.646868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.648328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:26.666067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:26.668056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 13:47:26.670142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:26.672031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 281/570] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:26.673019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.678876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.679659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.679951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.680228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.681427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=f3eacd1356f448cca079f57f683f37bf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:26.681792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.683259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63676e-609b-4b4f-975f-dca880f424c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.690579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 290/571] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/role_assignments?user.id=f3eacd1356f448cca079f57f683f37bf&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:26.691870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19437a55-87cf-4f9b-ac23-8646e4b6ba28 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.692165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19437a55-87cf-4f9b-ac23-8646e4b6ba28 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.692628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19437a55-87cf-4f9b-ac23-8646e4b6ba28 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.693891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 291/572] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:26.697347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19437a55-87cf-4f9b-ac23-8646e4b6ba28 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:26.811696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:26.829779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642b021-e9bd-446d-b866-c5603057c163 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.839942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:26.880376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19437a55-87cf-4f9b-ac23-8646e4b6ba28 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:26.894727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.896009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.896416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.898015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.899510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:26.900087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.902934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2374e-cb93-4c1d-92e0-123e83897dd7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.922675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 282/573] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:26.925727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.926440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.927022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.937928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:26.938614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.938896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.939201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.940063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:26.940418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:26.942469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd5e06-c4a5-4b3d-88a2-becef2c2da30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:26.948413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:26.950133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:26.972449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 292/574] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:26.972449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad325d9-5114-4064-a6a1-8908a1522ee4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:26.972449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad325d9-5114-4064-a6a1-8908a1522ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:26.972449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad325d9-5114-4064-a6a1-8908a1522ee4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:26.975887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 293/575] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:26.976642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad325d9-5114-4064-a6a1-8908a1522ee4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.043662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 283/576] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:27.044563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.108551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad325d9-5114-4064-a6a1-8908a1522ee4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.144456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:27.151329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:27.152061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.152335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.152638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.162625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:27.163891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:27.165215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2178743e-51e1-4f95-9c44-6a7a9834c89e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:27.180184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:27.181919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 294/577] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.183651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.184856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.194471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d2c93-c723-488b-a60b-a552de303cda None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.217000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:27.218270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:27.245143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:27.246296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.246590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.246845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.263174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:27.265085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:27.291154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:27.293301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:27.295110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:27.295110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:27.295110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:27.295110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:27.295110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 284/578] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:27.295110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.322146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 295/579] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:27.329600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.466905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c17a5-5b1d-4758-93ff-24569ec90f3d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.471068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc9ba8-3354-4da8-b4a1-0efdc762dc2c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.505114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:27.506363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:27.518729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:27.520021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.520516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.520865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.521060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:27.521691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.521978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.522255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.523059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:27.523367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:27.524871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:27.538551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:27.549250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 296/580] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:27.549250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:27.554628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:27.555598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:27.556425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:27.557623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 285/581] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:27.559616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.667085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a234f-7a16-4ed4-9213-f422a1550c9e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.720971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:27.721544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.721826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.722104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.722856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:27.723163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:27.724875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:27.747235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 297/582] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:27.747235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.756175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd868cca-4cf0-429e-a957-8be487ef2268 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.816272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:27.817553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:27.818027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:27.818500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:27.823843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:27.837215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:27.849092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:27.851002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:27.852655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:27.856948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 286/583] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:27.858777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:27.946629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052a794-c9d7-4e94-9e3b-c3d9670b4fa4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:27.964279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:28.013923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:28.015821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:28.016301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:28.016798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/e424ea6d03414857bc408e49b8df5429/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:28.045764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=e424ea6d03414857bc408e49b8df5429, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:28.047823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:28.062140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c4a6d-11b2-42de-baaf-c2cf03c2c046 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:28.079758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 298/584] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:27 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/e424ea6d03414857bc408e49b8df5429/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:28.082449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:28.152466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:28.154432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:28.155470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:28.156275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:28.167799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:28.168815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:28.172030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:28.197309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 287/585] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:28.199435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:28.294427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:28.317835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b40616-5a96-400e-9aa3-802a8810e4ae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:28.380963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:28.382325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:28.382795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:28.383250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:28.396414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:28.398091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578bd89-aad1-4de2-940a-aef04081fb6b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:28.399085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:28.417061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:28.434579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:28.436376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:28.438422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 299/586] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:28.440484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:28.452041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:28.453575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:28.454305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:28.455041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:28.456701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:28.457560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:28.460476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:28.478015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 288/587] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:28.480104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:28.657492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c396643-ed60-4804-ba25-b664b7198bf7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:28.691531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1c95c-cf3f-420c-84e3-403386a0a33f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:28.725501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:28.726904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:28.727818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:28.727992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:28.749913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:28.752645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:28.762384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:28.763836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:28.764657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:28.765437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:28.767461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:28.768359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:28.771325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:28.788274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:28.790043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:28.793175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:28.793175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:28.793175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:28.793175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:28.793175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 300/588] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:28.793175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:28.799307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:28.812608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 289/589] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:28.814838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:28.881445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:28.988281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27abcaf-ac6b-4b21-b207-4d69c0b8e52a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.019632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a632d-9f39-4130-b6c1-555b1adc4a70 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.047460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.049487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.049956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.050408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.065785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:29.067859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:29.082087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.083207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.083700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.084184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.085560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:29.086106 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:29.088686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c84e4-99f3-42d1-99cb-e67df293c4f3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:29.089863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:29.090720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:29.091585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:29.091585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:29.091585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:29.091585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:29.093243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 301/590] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:29.094217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:29.117641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 290/591] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:29.120344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a1a9d-866e-44b8-ad68-584eaa25e301 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.121025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a1a9d-866e-44b8-ad68-584eaa25e301 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.121621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a1a9d-866e-44b8-ad68-584eaa25e301 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.126564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 291/592] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:29.126564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a1a9d-866e-44b8-ad68-584eaa25e301 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:29.198423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:29.279202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:29.290543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cd99-e9a4-4827-9442-30b0e1f3894a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.294594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a1a9d-866e-44b8-ad68-584eaa25e301 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.346561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e117c-98c3-4b66-af4b-7bf1a64a2034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.348189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e117c-98c3-4b66-af4b-7bf1a64a2034 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.349024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e117c-98c3-4b66-af4b-7bf1a64a2034 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.349511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e117c-98c3-4b66-af4b-7bf1a64a2034 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/deb87accdcab4759add4b76958d151a8/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.350124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.351120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.351775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.352251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.353230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:29.353922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:29.356002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:29.378799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 292/593] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:29.378799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:29.384714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e117c-98c3-4b66-af4b-7bf1a64a2034 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=deb87accdcab4759add4b76958d151a8, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:29.387279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e117c-98c3-4b66-af4b-7bf1a64a2034 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:29.409428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 302/594] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:29 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/deb87accdcab4759add4b76958d151a8/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:29.412689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.413171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.413681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.447705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:29.449693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:29.578515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223824b9-d1fd-45e3-a8a3-e2301c7c2e5e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.612969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 303/595] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:29.614578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:29.662567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.663841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.664502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.664933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.686302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:29.689868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:29.696375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:29.724797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:29.726538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:29.728350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 293/596] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:29.729506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:29.762066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:29.822701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18f665-7ad1-4d0d-a682-c2da5411f83b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.885525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.886840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.887321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.887783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:29.901815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:29.904766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:29.919617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c06bc0-5f2c-4c97-8b1d-cb54130afb03 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:29.934206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:29.936037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:29.937921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 304/597] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:29.939649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:29.984516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:29.985836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:29.986850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:29.987318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:30.007512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:30.021915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:30.025934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 294/598] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:30.029013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:30.133088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80b52f-ea94-42fd-bc3c-bc17c503882e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:30.157164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:30.178973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:30.232606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:30.233454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f14cc-4a1b-4650-9853-8c145986b30f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:30.234613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:30.235389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:30.236776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:30.240428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:30.241406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:30.244491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:30.296655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 305/599] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jul 15 13:47:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:30.300693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:30.332239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:30.333960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:30.334742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:30.335529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:30.337303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:30.338153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:30.341219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:30.384967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 295/600] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Jul 15 13:47:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:30.387057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:30.511196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e83a6d-8771-4d70-aee6-d394dded7893 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:30.583779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8b80c-0b24-4a56-b8b6-bd7d61b45e07 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:30.585876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:30.587103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:30.587607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:30.588284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:30.591091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:30.591091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:30.593357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:30.620361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 306/601] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:30.620361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:30.652318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:30.666600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:30.668473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:30.669269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:30.670076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:30.674292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:30.693831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:30.696955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:30.733044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:30.734812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:30.736624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:30.736624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:30.736624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:30.736624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:30.747539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 296/602] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:30.749443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:30.834301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87cfec-ba2d-41a4-a3a3-e135b20571e1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:30.906415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:30.907669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:30.908743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:30.909235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:30.931725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:30.932943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9fb0b-38fc-4c6c-a066-b307d2a771ac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:30.934701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:30.972941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:30.974820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:30.976582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:30.976582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:30.976582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:30.976582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:30.981153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 307/603] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:30.982757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.009217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.010840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.011657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.012172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.024934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.027223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:31.046809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:31.047780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:31.049121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:31.049121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:31.049121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:31.049121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:31.049561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 297/604] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:31.050717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.098149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:31.124772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:31.173224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20192c-0382-4199-b64e-f8d563bfe4d8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.179018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607fedf-74f7-4c07-9660-c6ee00a82dff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.217779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.218492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.218793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.219069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.219815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:31.220131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.221627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479c6e5-d8c0-40e9-8e5a-da3db0f083bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.229484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.230349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.230619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.230885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.231768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:31.232077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.233756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.239027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 308/605] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:31.244575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c969-12b8-4178-90f0-df0a9d7ed274 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.244876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c969-12b8-4178-90f0-df0a9d7ed274 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.245159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c969-12b8-4178-90f0-df0a9d7ed274 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.246849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 309/606] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:31.255643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 298/607] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:31.255643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.256000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c969-12b8-4178-90f0-df0a9d7ed274 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.385897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:31.393835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d401-2856-42e9-b901-0c010aae8079 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.405803 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c969-12b8-4178-90f0-df0a9d7ed274 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.444430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:31.456256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.457098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.457446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.457710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.459316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.460486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.460998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.461499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.462934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:31.463500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.466209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.466750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.468427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-adf2558e-bd37-45b2-9ed1-1a027d806eb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:31.487985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:31.498000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 299/608] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.499456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.502118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.502350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 310/609] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:31.502350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.516054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:31.517648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:31.633136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 300/610] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:31.635879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.643766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26933cad-e90c-457c-a21b-d0c5e128ba14 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.698028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.698855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.699132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.699404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.708280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.709909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:31.738815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:31.740780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 311/611] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:31.742546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.801415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:31.815590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:31.826539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d216d-bafa-48e9-a675-8cd7a01c44bb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.876567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.877508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.878061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.878454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.879370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:31.879800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.881848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.891331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ad967-f477-4880-b65a-598687ff980c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:31.899143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 301/612] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:31.901044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:31.953328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:31.954405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:31.954887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:31.955366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:31.965019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:31.966661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:31.986920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:31.988858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:31.990613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 312/613] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:31.991752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.111448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc864597-b9c5-437a-acb2-6dd72bb434c6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.151921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c0aa3-828b-41ae-b29a-227485af102c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.169070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:32.184643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.185945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.186408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.186879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.205678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:32.213603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:32.215387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 302/614] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:32.217045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.222001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.224814 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.226783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.227427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.229200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:32.229895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:32.233944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:32.234229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:32.259196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 313/615] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:32.261549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.369340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec7254-afc2-4437-8529-3fd5fb4d6c98 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.431076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.432800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.433409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.434037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/f0e94eecf28f42468e14bbd0c4eb3777/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.447216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a40a18-54c7-40e2-b1b5-42ca28ff972e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.452433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=f0e94eecf28f42468e14bbd0c4eb3777, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:32.453840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:32.472123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 303/616] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:32 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/f0e94eecf28f42468e14bbd0c4eb3777/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:32.473022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.513515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.514870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.515981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.516203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.518434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:32.519006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:32.521716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:32.561555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 314/617] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jul 15 13:47:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:32.561555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.572118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:32.618304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:32.659161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a6e0-b386-4d58-812d-f6bc84029dcd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.725272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.726180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.726470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.726753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.727576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:32.727884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:32.729541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df9bc2-3944-46c4-ac80-0635219d9e96 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:32.740661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 304/618] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:32.743050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aba1a-4b8c-49f7-b13c-84504472f0b1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.743550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aba1a-4b8c-49f7-b13c-84504472f0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.744049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aba1a-4b8c-49f7-b13c-84504472f0b1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.747016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 305/619] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:32.750095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aba1a-4b8c-49f7-b13c-84504472f0b1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.762382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa04b9a-8574-40c3-82d2-de6cfc2634ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.833284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.834919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.835457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.835975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/393c4bea16074727958ea40ab497948d/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:32.862273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=393c4bea16074727958ea40ab497948d, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:32.865522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:32.890817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 315/620] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:32 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/393c4bea16074727958ea40ab497948d/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:32.893605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:32.932728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aba1a-4b8c-49f7-b13c-84504472f0b1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:32.975057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:32.995553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:32.996271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:32.996612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:32.996860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.014594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:33.016083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-97798ccb-db25-47c3-9260-f0d2290b338a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:33.037191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:33.038948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 306/621] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:33.041415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.072682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:33.077857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:33.079434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:33.127118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcf717-cb3e-4a45-baed-8b5ae7994f89 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:33.168429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 307/622] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:33.168429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:33.209026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:33.210541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:33.211063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:33.211565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.235478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:33.238684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:33.275693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:33.277462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:33.279099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:33.279099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:33.279099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:33.279099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:33.279099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 316/623] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:33.280360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:33.317819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd49915-ffeb-4e23-b9f1-3738e5b6685d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:33.321497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:33.393305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:33.394918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:33.395690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:33.396498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.398160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:33.398971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:33.403355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:33.446759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 308/624] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:33.448764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:33.543462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ae78-09d0-418a-9f7d-e4f6ee8df9fc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:33.588067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:33.610348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:33.611699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:33.611977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:33.612240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.635644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:33.643517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:33.645326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:33.646902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:33.650158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 317/625] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:33.651165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:33.793451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba3821-6875-4129-94d0-75d3931bfd46 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:33.826700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c827067-dc33-4833-82a7-9e9dc1f01054 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:33.852084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:33.852947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:33.853225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:33.853492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.862492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:33.864001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:33.881461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:33.883206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:33.884913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:33.887847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 318/626] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:33.889430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:33.940587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:33.943532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:33.946164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:33.947019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:33.948488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:33.949048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:33.951675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:33.970861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:33.981072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 309/627] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:33.982695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.035878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:34.055622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab381e9-0cfa-49cb-8956-7fd9677302f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.099980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.100983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.101908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.101908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.102574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:34.103165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.104696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:34.120019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 319/628] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:34.121147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.252561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a8c5-fe87-4c02-9ecf-e786d0425858 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.262651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5085560-150c-427c-ad01-4e48764f660a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.266075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:34.295046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.295545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.295939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.296454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.305925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.308096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:34.323224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:34.325347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 320/629] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:47:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:34.334715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.343342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.344163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.344582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.344960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.346507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:34.346914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.348516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:34.371864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 310/630] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:34.373798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.464668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d83a-e302-4abb-8d92-b383c2187a5c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.503182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.504302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.504551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.504943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.517654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.519304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:34.548060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:34.550169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:34.552008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:34.552008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:34.552008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:34.552008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:34.552008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 321/631] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:34.552567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.558613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:34.607014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3c2b2-72db-47fc-9b7c-1d7f7dbe40b4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.613893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:34.666366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.667370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.667795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.668256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.669345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:34.669814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.671424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:34.695851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 311/632] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:47:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:34.697208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.729320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5567b9-34ed-47b9-9548-2781630caf30 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.774982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.776757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.777585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.779257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/f3eacd1356f448cca079f57f683f37bf/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.825579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=f3eacd1356f448cca079f57f683f37bf, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.828715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:34.870490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 322/633] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:34 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/f3eacd1356f448cca079f57f683f37bf/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:34.870490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:34.871957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30ed98-3e25-43b9-bf94-6716ffdd9557 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:34.965499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:34.967103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:34.967849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:34.968690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:34.970951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:34.971820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:34.980338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0daca0-e448-4eca-a6a5-f122cfb712a4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:35.008131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:35.015244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 312/634] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:35.017549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e15342-db7c-473a-bbee-97c0eeaf4ad6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.018389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e15342-db7c-473a-bbee-97c0eeaf4ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.018896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e15342-db7c-473a-bbee-97c0eeaf4ad6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.025348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 313/635] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:35.026478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e15342-db7c-473a-bbee-97c0eeaf4ad6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:35.057423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:35.119456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7baba-d67c-4bdb-a8f0-e6d14c0e2d60 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:35.174203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e15342-db7c-473a-bbee-97c0eeaf4ad6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:35.200427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:35.201796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.202328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.202825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.235404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:35.237830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.238440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.239211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.251336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:35.253276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-239bd833-5197-46f8-a05f-fd6277e4eeb2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:35.269539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:35.272198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 314/636] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.274784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.277022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.294973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:35.295956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:35.346974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:35.351264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:35.366271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:35.388427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 315/637] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:35.388427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:35.542744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:35.589317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afe035-838f-4132-b651-51ef5e5fb020 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:35.605211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 323/638] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:35.607252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:35.636240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:35.636958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.637239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.637512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.639033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:35.639353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:35.640872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:35.668497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 316/639] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jul 15 13:47:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:35.670624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:35.738538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:35.807940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddc64a-63fb-464c-889c-c9bc1b9a3697 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:35.818861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8b330-0f1a-4aa6-8540-a07017054bc0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:35.853878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:35.855655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.856193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.856719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.873306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:35.876174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:35.885398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:35.886595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:35.887073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:35.887547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:35.902774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:35.911721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:35.912996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:35.913809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:35.916524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 324/640] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:35.917695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:35.925495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:35.926497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:35.927382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:35.927382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:35.927382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:35.927382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:35.928022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 317/641] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:35.929307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:35.966066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.058067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baece8-7cfc-4915-a091-d67aa4f560ae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.102843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac2b77-1375-4a3e-909e-f5d1e52d7f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:36.104336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2b77-1375-4a3e-909e-f5d1e52d7f3d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.104834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2b77-1375-4a3e-909e-f5d1e52d7f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.105303 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac2b77-1375-4a3e-909e-f5d1e52d7f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.136141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.144060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fac386-bf1e-48c3-85af-a14c8b0f824c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.196962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac2b77-1375-4a3e-909e-f5d1e52d7f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:36.199086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac2b77-1375-4a3e-909e-f5d1e52d7f3d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:36.220112 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:36.221354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.221956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.222498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.224233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:36.224896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:36.227638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:36.241122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.252786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 318/642] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:36.254009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:36.362797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 325/643] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:36.362797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4884c6-6b7f-4c21-bc12-8fbbf214e9fc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.362797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4884c6-6b7f-4c21-bc12-8fbbf214e9fc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.362797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4884c6-6b7f-4c21-bc12-8fbbf214e9fc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.364229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 326/644] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:36.366274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4884c6-6b7f-4c21-bc12-8fbbf214e9fc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:36.418870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf80509-5681-494a-8b25-ffde1918f947 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.477257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:36.478387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.478897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.479380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.480562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:36.480953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:36.483029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:36.509884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 319/645] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:36.509884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:36.536667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4884c6-6b7f-4c21-bc12-8fbbf214e9fc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.552854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.580233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.591184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:36.592644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.593151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.593707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.616719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:36.619383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-31a9dd9f-4886-4943-a625-96709bd264fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:36.645527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:36.647288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 327/646] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.649290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.667701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:36.668931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:36.687308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e65de-5e60-4652-a071-96c6e91b2747 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.737026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:36.737848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.738812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.739077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:36.753178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:36.754639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:36.778349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:36.780021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:36.782243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:36.782243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:36.782243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:36.782243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:36.782243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 320/647] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:36.782243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:36.788507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 328/648] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:36.789607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:36.882355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.924418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033f6e7-cfce-4c75-9b49-12d58ce4716a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.957604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af43e1-042e-4b69-9f9a-805f227c8586 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:36.976191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:36.994234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:36.995586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:36.996081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:36.996599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.002988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.003661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.003939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.004213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.004982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:37.005281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.006660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.008967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.011665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.026969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 329/649] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:37.026969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:37.036332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:37.038890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 321/650] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:37.043432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.203493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298bd55-5c7e-4c88-8073-28940b94de74 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.223959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53c72f-0cf7-4a7b-ac30-1ce14416806b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.264023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.265089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.265553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.266011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.267331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:37.267807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.270050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.288930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.290492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.291138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.291758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 330/651] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:37.292001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.293250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.297040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:37.314490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:37.329100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:37.331457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 322/652] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:37.340921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.363047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:37.471352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb511ea8-6adb-4d38-8457-808eb5db7c74 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.495272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3120-dd2f-4020-9d57-cf1e33a9a8ff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.521894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.522866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.523325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.523786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.524959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:37.525472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.527968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.550670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 323/653] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:37.552033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.563353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.567185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.567888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.568551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.584371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.587297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:37.619691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:37.621661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 331/654] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:37.628570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.650891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:37.740840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080c427-11ba-494d-90d0-517172f9449c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.808858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.809587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:37.810467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.811061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.811707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.822711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d9f6e-30af-4c91-b591-0868789adc12 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.825525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.828419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:37.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:37.857341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:37.859386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 324/655] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:37.860063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.888604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:37.889856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:37.890403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:37.890940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:37.892659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:37.893220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:37.895810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:37.911758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 332/656] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:37.913041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:37.972400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c988815-3a9e-4a19-8380-3feb89116bcd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:37.988919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:38.013147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.013717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.013997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.014270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.015139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=e424ea6d03414857bc408e49b8df5429) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:38.015448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.016946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1b96e-7e8e-4c26-a927-9c93c3f59471 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.023924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 325/657] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/role_assignments?user.id=e424ea6d03414857bc408e49b8df5429&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:38.024941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaabce-47d6-48f5-ac45-ce8fa211b863 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.026458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaabce-47d6-48f5-ac45-ce8fa211b863 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.026458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcaabce-47d6-48f5-ac45-ce8fa211b863 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.032969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 326/658] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:38.032969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcaabce-47d6-48f5-ac45-ce8fa211b863 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:38.081382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646162-3efe-4c01-a633-c4ffd9ad1017 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:38.147637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.148922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.149412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.149933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.151640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:38.152208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.155189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.198249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 333/659] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jul 15 13:47:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:38.198249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:38.215531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:38.221274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcaabce-47d6-48f5-ac45-ce8fa211b863 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:38.281978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.284067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.284067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.284067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.284626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:38.284895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.286894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa4b85-98a8-404d-be18-5807e519d78e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.303629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 327/660] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:38.305105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.305402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.305675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.325082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:38.326346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:38.364937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d299-3861-4778-8078-6cb9bf42f427 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:38.388563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:38.412875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.413894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.414332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.414766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.432978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.434454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:38.455285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:38.457066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:38.458832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:38.458832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:38.458832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:38.458832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:38.458832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 334/661] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:38.458832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:38.471547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 328/662] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:38.478468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:38.575981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:38.618563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc624336-621b-46f5-82af-8a75997a3a50 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:38.647716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c25b6-985f-4d4e-ab93-488a3aaf683d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:38.669671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.670605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.671026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.671449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.672527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=deb87accdcab4759add4b76958d151a8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:38.673003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.674863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.692432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 329/663] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:38 2022] GET /identity/v3/role_assignments?user.id=deb87accdcab4759add4b76958d151a8&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:38.692432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:38.718314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.719574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.720313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.720888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.756813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:38.842069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.844631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.890504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d35da-7b31-44cd-bd04-70567c3e2343 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:38.931501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:38.932693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:38.933182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:38.933729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:38.934681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:38.935243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:38.936948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:38.951389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 330/664] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:38.956504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:38.962019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:39.008561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 335/665] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:39.010151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.076883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:39.090851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce11b11-fb95-4024-aba8-f00530d47569 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:39.140420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:39.141157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:39.141430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:39.142236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:39.156073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:39.165038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:39.166810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 331/666] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:39.169151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.219867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c571c6-0073-4344-b7c2-bef574a8a64c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:39.298399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:39.299245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:39.299527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:39.299811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:39.309435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:39.310947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:39.328694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:39.331074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 336/667] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:39.333016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.355042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8f367-d3a8-4ba6-b528-a9749e26b387 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:39.394244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:39.423527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:39.425371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:39.425811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:39.432437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:39.456848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:39.458465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:39.480239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:39.481654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:39.482891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:39.482891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:39.482891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:39.482891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:39.482891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 332/668] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:39.483450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.488681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:39.527443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b352ecd-81d0-4183-b428-37cc672a2c6e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:39.597629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:39.598907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:39.599331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:39.599754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:39.601113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:39.601484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:39.603826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:39.619598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 337/669] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:39.621103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.694357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53383-2019-40c6-bab1-89ea8f89bc9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:39.772831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:39.775745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:39.776536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:39.777204 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:39.779050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:39.779771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:39.782973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:39.809253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:39.826758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 333/670] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:39.828668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.842029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546f069-27ac-413c-9b15-1b8d970cb51b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:39.895384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:39.896765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:39.897235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:39.898703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:39.899716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:39.900180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:39.901779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:39.917834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 338/671] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:39.918982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:39.962909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:40.017847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc0b88-ff91-4e92-9a2c-26e24154f3de None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.064500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.065682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.066118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.066540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.080690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.082411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.085449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c08fc7-1ce3-4e80-9ad1-00f1f9518abd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:40.103279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:40.104240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:40.105150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:40.105150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:40.105150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:40.105150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:40.107598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 334/672] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:40.108804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:40.125913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.126944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.127431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.127743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.179721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:40.214035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.215794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.239261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f849f4a-acaa-49f9-a1ec-13d80956a380 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.256929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:40.301867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.303285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.303780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.304272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.318614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.321557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.336159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 339/673] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:40.337176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:40.359898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:40.361081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:40.362203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:40.364576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 335/674] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:40.365935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:40.397549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:40.467423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe84ba-cf5b-4aec-a555-f22ab6cac3b9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.509530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a6726-9ce6-401c-a5dc-16d7c796673d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.513855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.514681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.515086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.515488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.516366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:40.517393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.519298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780d3e6-bdfa-4d30-b9e5-1dd0c135d478 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.539655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 340/675] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:40.542519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619343-bc50-4a65-a9ed-9b3892e20901 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.543018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619343-bc50-4a65-a9ed-9b3892e20901 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.543501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619343-bc50-4a65-a9ed-9b3892e20901 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.550211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 341/676] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:40.550211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619343-bc50-4a65-a9ed-9b3892e20901 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:40.572489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.573705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.574164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.574619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.594096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.596962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:40.626965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:40.627912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:40.628810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:40.628810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:40.628810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:40.628810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:40.628810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 336/677] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:40.629919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:40.666930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:40.722872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619343-bc50-4a65-a9ed-9b3892e20901 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.740288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:40.781484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.782326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.782480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.782846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.783674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:40.783987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.785270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01d05f-faf5-4604-81c7-62b51fc19b6d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:40.785512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad409e-3fc7-4cb5-803b-670b23b93228 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.799212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 342/678] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:40.800334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.800658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.800939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.824515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:40.826071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:40.842424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:40.843416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:40.843701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:40.843982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:40.855524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:40.857247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:40.873302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:40.874880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:40.877181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:40.879695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 337/679] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:40.880666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:40.928637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 343/680] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:40.931789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.011916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:41.036106 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82775907-bd2d-4866-b754-e804b0e3f1b1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.069753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:41.074524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51209c-b4b6-4017-a38b-b146155b95e6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.089624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.090640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.091044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.091438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.092498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:41.093340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:41.095601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:41.118479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 338/681] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:41.118479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.129306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.129980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.130292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.130537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.131363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=f0e94eecf28f42468e14bbd0c4eb3777) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:41.131681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:41.133189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:41.140371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 344/682] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:40 2022] GET /identity/v3/role_assignments?user.id=f0e94eecf28f42468e14bbd0c4eb3777&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:41.141258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.289697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf9a5e-0b2f-45ff-b0f7-9703d43c181d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.331010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a42b15-3757-4383-b503-2e214a997012 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.350104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.351190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.351702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.352176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.353534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:41.354045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:41.356616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:41.390101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 339/683] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jul 15 13:47:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:41.390101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.390389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87725e21-66a0-4593-9c71-302a0e51a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.390389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87725e21-66a0-4593-9c71-302a0e51a03c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.390969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87725e21-66a0-4593-9c71-302a0e51a03c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.391503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87725e21-66a0-4593-9c71-302a0e51a03c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.395556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:41.413032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87725e21-66a0-4593-9c71-302a0e51a03c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-87725e21-66a0-4593-9c71-302a0e51a03c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:41.427837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:41.428872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:41.429692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 345/684] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:41.431603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25388257-4a0a-48aa-a9cb-4ed2b0d77e27 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.432099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25388257-4a0a-48aa-a9cb-4ed2b0d77e27 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.432632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25388257-4a0a-48aa-a9cb-4ed2b0d77e27 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.435878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 346/685] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:41.437496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25388257-4a0a-48aa-a9cb-4ed2b0d77e27 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.473336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:41.542733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643b05a-9923-4192-9277-da1a37226988 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.585623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25388257-4a0a-48aa-a9cb-4ed2b0d77e27 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.602108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.603385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.603970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.604383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.621692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:41.623257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:41.638811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:41.639977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 340/686] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:41.642346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.647782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.649005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.650579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.650579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.738307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:41.739990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:41.759707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:41.784434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:41.821927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5fe82-f8bb-426e-94d5-0692c1d8e0bc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:41.880830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:41.881911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.882376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.882828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.883765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:41.884282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:41.886735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7c7cd-2a37-482e-bf8a-376c74563bf0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:41.901843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 341/687] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:41.904236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:41.904937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:41.905630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:41.932462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 347/688] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:41.933883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:41.938269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:41.941020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:42.065899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 342/689] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:42.068692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.079971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcef70-ff82-456c-9e41-5272e6183910 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.122723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:42.142509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:42.143369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:42.143787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:42.144219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:42.145266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:42.145780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:42.147471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:42.166982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 348/690] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:42.172752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.187982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:42.258644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddb0c6-d735-4cf4-9674-c3dfea994568 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.322835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:42.323812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:42.324538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:42.324818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:42.345585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:42.348667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:42.358591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17c91a-7321-4194-989a-a844641e4078 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:42.389988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:42.391963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:42.393981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:42.393981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:42.393981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:42.393981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:42.399399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 343/691] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:42.400859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.435428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:42.436724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:42.437184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:42.437634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:42.455955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:42.458552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:42.491983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:42.493912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:42.496777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:42.496777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:42.496777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:42.496777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:42.496777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 349/692] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:42.496777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.557378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:42.575290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c47010-2c1c-48e2-ac5d-d5240de58b51 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.588357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:42.623151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:42.624454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:42.624906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:42.625310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:42.638267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:42.640091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:42.664173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:42.666126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:42.675346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:42.675346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:42.675346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:42.675346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:42.675346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 344/693] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:42.675346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.699464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ffc2b-5016-4bd0-b349-e18b6a042395 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.754754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:42.755552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:42.755909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:42.756289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:42.757302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=393c4bea16074727958ea40ab497948d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:42.757701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:42.759333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:42.776443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 350/694] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/role_assignments?user.id=393c4bea16074727958ea40ab497948d&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:42.776443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.823877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb168572-ab3c-456d-820b-86d6e5f32332 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.889033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83979010-c87a-4838-bd51-077be2050487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:42.892802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83979010-c87a-4838-bd51-077be2050487 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:42.893718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83979010-c87a-4838-bd51-077be2050487 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:42.894493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83979010-c87a-4838-bd51-077be2050487 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:42.897481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83979010-c87a-4838-bd51-077be2050487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:42.898624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83979010-c87a-4838-bd51-077be2050487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:42.903245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83979010-c87a-4838-bd51-077be2050487 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:42.945143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 345/695] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:42.947408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83979010-c87a-4838-bd51-077be2050487 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:42.968669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:42.981892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd855ffe-0865-4017-b174-80e69f32e4b6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:42.984479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:43.052336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.053220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.053605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.053984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.054847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:43.055244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.056751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.084013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 351/696] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:43.086739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:43.094963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83979010-c87a-4838-bd51-077be2050487 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.136799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.137618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.137903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.138173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.138910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:43.139292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.141486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.172425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 346/697] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:43.172425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:43.308033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891c70-7bb4-4143-b77e-cb0131082f03 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.344756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:43.377355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.379239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.379946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.380744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.385798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a77e3a-d79a-476f-98d6-bf8d6cc2bbfb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.396622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.399658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:43.430649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:43.432607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:43.434434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:43.439229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 352/698] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:47:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:43.441970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:43.456936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.457771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.458056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.458337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.473049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.475871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.486614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:43.495751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:43.496747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:43.497604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:43.500339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 347/699] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:43.502212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:43.619726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6e549-4c4d-46e8-b639-776e4ae9aaaa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.662215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffffe4b-f2b2-43d5-858c-270ce1f9375b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.666796 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.667646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.668037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.668434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.669439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:43.669811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.671356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.683136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:43.702930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 348/700] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:47:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:43.704567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:43.736084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3bdac-560d-4a50-9af9-e94bee57af24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.737386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3bdac-560d-4a50-9af9-e94bee57af24 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.737806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3bdac-560d-4a50-9af9-e94bee57af24 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.738215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3bdac-560d-4a50-9af9-e94bee57af24 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.739248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3bdac-560d-4a50-9af9-e94bee57af24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.741104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3bdac-560d-4a50-9af9-e94bee57af24 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.765870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 353/701] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:47:43 2022] POST /identity/v3/users => generated 254 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:43.768293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407cb20-5e72-4f4b-96aa-e0360e5e1b4e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.768879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407cb20-5e72-4f4b-96aa-e0360e5e1b4e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.769413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407cb20-5e72-4f4b-96aa-e0360e5e1b4e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.772054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 354/702] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:43.773254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407cb20-5e72-4f4b-96aa-e0360e5e1b4e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:43.887941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bde216-4d6a-4772-9c64-483ee8b1725f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.913042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:43.941508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407cb20-5e72-4f4b-96aa-e0360e5e1b4e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:43.952762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.953577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.954111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.954649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.956411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:43.956601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:43.959412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbfb87-bdbf-4c34-95c3-72ae55a101b5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:43.992352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 349/703] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:43.994300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:43.994889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.995431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.995675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:43.996118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:43.996314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:43.996840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:44.024526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:44.026129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:44.103627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:44.105594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:44.111400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:44.139178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 350/704] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:44.140865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:44.232540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:44.275766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 355/705] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:44.281107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:44.288646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda557c-ea12-47e5-a3fc-f744d836bf9e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:44.334793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:44.336539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:44.336825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:44.337101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:44.337962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=f3eacd1356f448cca079f57f683f37bf) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:44.338270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:44.339738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:44.348247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 351/706] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/role_assignments?user.id=f3eacd1356f448cca079f57f683f37bf&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:44.349500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:44.428854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:44.480003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7be95b-05e6-45e2-be45-681b3a22d4f3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:44.525882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:44.526620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:44.528272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:44.528574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:44.557493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:44.558995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:44.577607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b91b-8f14-424b-ad6f-5205395bb82e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:44.600679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:44.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:44.602552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:44.602552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:44.602552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:44.602552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:44.613378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 352/707] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:44.618362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:44.647251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:44.648108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:44.648403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:44.648709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:44.649466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:44.651718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:44.669082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:44.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:44.671702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 356/708] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:44.679343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:44.755300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:44.794553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9967472-7ed3-4d8e-9841-3a9629d90896 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:44.800369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:44.844023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:44.844823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:44.845106 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:44.845386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:44.846272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:44.846575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:44.848126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:44.865087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 353/709] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:44.878876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:44.892595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8103e-415f-4f06-97bc-63dfc886d4f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:44.942959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:44.943832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:44.944225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:44.944635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:44.945472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:44.945885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:44.947418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:44.958723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 357/710] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:44.960168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:45.112072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c7481-1594-4c8a-94a6-e14d60432075 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:45.116904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496e078-1961-455e-91fb-5660e4087293 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:45.119854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:45.170750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:45.171678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:45.172332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:45.172332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:45.181714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:45.183342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:45.198832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:45.200749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:45.202717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:45.207225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 354/711] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:45.209718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:45.224062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:45.225402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:45.225887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:45.226375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:45.227838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:45.228372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:45.231078 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:45.243646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:45.264421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 358/712] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jul 15 13:47:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:45.359165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff754be-0af7-46e1-9acf-47815de54b69 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:45.413894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:45.414943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:45.415500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:45.416003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:45.417108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:45.417632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:45.419308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bfa01c-dbb5-4d57-b7dc-3b377affe2c1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:45.438911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 355/713] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:45.446031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:45.720702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b86d9-f6ef-4ff6-ab73-91e5afe5d613 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:45.774068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc2be6-8cbf-4c7e-adc0-622ced4cfcfa None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:45.774402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc2be6-8cbf-4c7e-adc0-622ced4cfcfa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:45.774630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc2be6-8cbf-4c7e-adc0-622ced4cfcfa None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:45.776356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 356/714] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:45.789773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:45.790290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:45.790802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:45.791594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:45.793051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:45.793817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:45.794449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:45.795534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=e424ea6d03414857bc408e49b8df5429) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:45.796106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:45.797858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:45.812716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 359/715] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:45 2022] GET /identity/v3/role_assignments?user.id=e424ea6d03414857bc408e49b8df5429&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:45.849741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:45.850980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:45.968951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 357/716] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:46.030308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:46.073072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:46.153388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57849170-0abc-4aa4-b987-8cb59d103f47 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:46.195480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:46.196306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:46.196623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:46.196919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] PATH_INFO: `/v3/services/network` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:46.197567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:46.198985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:46.212730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:46.214672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:46.216301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 360/717] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:46 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:46.233605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:46.257859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:46.386882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de132ec-83a2-4f50-a1b1-f12dd51b1c96 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:46.426749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:46.427430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:46.427710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:46.427987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:46.428745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] RBAC: Adding query filter params (name=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:46.429052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:46.430457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491a0f2-1610-43bb-825a-eec8b46b9d6a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:46.435966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 358/718] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:47:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:46.445416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:46.570985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:46.578476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f6a90-076b-4f42-a827-bdedb5cb7a43 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:46.616020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:46.616726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:46.617019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:46.617286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:46.618020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] RBAC: Adding query filter params (type=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:46.618327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:46.619710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e816d-56a5-4716-a745-ee60fd7ab9da None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:46.625145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 361/719] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:47:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:47.127034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31671234-8f80-439b-aa56-d45be7e3f32a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:47.127912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31671234-8f80-439b-aa56-d45be7e3f32a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:47.127912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31671234-8f80-439b-aa56-d45be7e3f32a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:47.129339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 359/720] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:47.134825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28496c6-c26f-476d-8463-7395312f660d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:47.135604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28496c6-c26f-476d-8463-7395312f660d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:47.135604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28496c6-c26f-476d-8463-7395312f660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:47.162516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e28496c6-c26f-476d-8463-7395312f660d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:47.163736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e28496c6-c26f-476d-8463-7395312f660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:47.263605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 362/721] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:47.286031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f1c5a-7486-434e-86ff-0e1fe2850bf8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:47.286468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f1c5a-7486-434e-86ff-0e1fe2850bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:47.286624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f1c5a-7486-434e-86ff-0e1fe2850bf8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:47.288026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 360/722] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:47:47.293861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:47.294237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:47.294487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:47.299285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f1c5a-7486-434e-86ff-0e1fe2850bf8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:47.316876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:47.318099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:47.436036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 363/723] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:47.458392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:47.471478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:47.485646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f1c5a-7486-434e-86ff-0e1fe2850bf8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:47.537042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:47.545222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:47.546058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:47.546351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:47.546642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:47.566591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:47.583057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:47.585584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:47.587295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:47.592803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 361/724] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:47.593777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:47.686419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f0d25-5c44-4731-af06-7604b2bcf923 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:47.754467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:47.756365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:47.757206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:47.757943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:47.778643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:47.800538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:47.802368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:47.804179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:47.810163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 364/725] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:47.818257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:47.947841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b4204-2e10-491b-8395-1bba2e259788 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.004956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:48.045716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:48.047227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.047979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.048973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.050925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:48.051776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:48.054555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:48.066937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:48.082820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 362/726] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:48.084042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e32e3-6c4e-4bc2-8e91-6cc156ede526 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.089365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:48.223086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:48.225583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.229206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.230086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.231886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:48.234498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:48.237592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:48.262802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 365/727] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:48.270456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:48.343486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dad12c-e537-4238-94c0-c2a1d956e749 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.384573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:48.385388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.385672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.385950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.398584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:48.400086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-621635c4-9e9c-4433-8d38-7647ccedd347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:48.419866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:48.422155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 363/728] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761de1c9-c242-4166-9064-d01afc7ad43b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761de1c9-c242-4166-9064-d01afc7ad43b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.424094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761de1c9-c242-4166-9064-d01afc7ad43b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.429571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 364/729] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:48.430540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761de1c9-c242-4166-9064-d01afc7ad43b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:48.478705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d93b8-8926-44c8-bd19-58ad60ddd773 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.495036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:48.542968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:48.544067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.544525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.544990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.555547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761de1c9-c242-4166-9064-d01afc7ad43b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.562055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:48.567839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:48.572280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:48.601249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:48.602183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.602617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.603041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.603946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:48.604462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:48.606015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0aa3264d-ebc6-4669-91ee-82571f343079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:48.606500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:48.607389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:48.608244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:48.608244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:48.608244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:48.608244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:48.610719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 366/730] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:48.613112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.613756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.614471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.625472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 365/731] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:48.627340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:48.645724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:48.648007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:48.748930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa88a8e-b76b-4bc3-90cf-7f76ebbde22c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.764664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:48.784472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 367/732] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:48.784472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:48.808577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:48.809521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:48.809948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:48.810389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:48.811296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:48.811780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:48.814034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:48.829188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 366/733] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:48.834730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:48.961360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7870c-4ba3-4742-9ceb-1844e84191e4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:48.975603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6413a-ee8d-43de-b326-c633744612a0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.000029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:49.018343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.019652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.020144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.020649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.033732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.036359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.037192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.038723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.039324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.040014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.053802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.056838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4b9a3094-7bc3-44b4-8933-48c0bdde50ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:49.063761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:49.065654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:49.068335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 368/734] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:47:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:49.070753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7019-b38a-4fa9-ba96-a3a2b3764be3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.070753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7019-b38a-4fa9-ba96-a3a2b3764be3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.070753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e7019-b38a-4fa9-ba96-a3a2b3764be3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.073071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 369/735] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:49.073798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7019-b38a-4fa9-ba96-a3a2b3764be3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:49.083762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:49.085903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:49.087239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:49.088777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 367/736] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:49.089876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:49.135900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:49.254434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e7019-b38a-4fa9-ba96-a3a2b3764be3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.259314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749cf95-fcb8-4f4d-be15-2484a3767ca2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.311961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c28fc-dd66-4b0f-9ca7-657094caec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.313691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c28fc-dd66-4b0f-9ca7-657094caec19 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.314363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c28fc-dd66-4b0f-9ca7-657094caec19 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.315036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c28fc-dd66-4b0f-9ca7-657094caec19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.316437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.317469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.317921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.318387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.319888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:49.320346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.322896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.334822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c28fc-dd66-4b0f-9ca7-657094caec19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-bf6c28fc-dd66-4b0f-9ca7-657094caec19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:49.349535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:49.351229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:49.352688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 370/737] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:49.355769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.355769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.355769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.361446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 368/738] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jul 15 13:47:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:49.362874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:49.379926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:49.382902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:49.447412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:49.504580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 371/739] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:49.504580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:49.517258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:49.533376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453b953-b4d2-4327-bc83-9db78f659fb6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.593328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.594230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.594544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.594999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.596536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:49.597034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.600044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.633709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 369/740] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jul 15 13:47:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:49.635170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:49.679952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd562d86-98b7-45ee-8fa7-7ea049f194c9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.726677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.727409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.727696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.727971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.728787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:49.729094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.731581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.752869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 372/741] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:49.752869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:49.789781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6d99-7bfd-4e39-b43e-c6a331733e81 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.798794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:49.834094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.834850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.835033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.835315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.858014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:49.889443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2319bea-9000-4bcf-9c30-75969f15910f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:49.945492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:49.946168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:49.946451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:49.946715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:49.947457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.949010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.951782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:49.953935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:49.966974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 373/742] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:47:49 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:49.967916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:50.118318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:50.138306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48011b64-3487-48a9-a496-93e84d3f03f4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:50.159602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 370/743] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:50.164417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:50.221138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6c0ad-7111-41d1-b624-716639d7b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:50.222193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6c0ad-7111-41d1-b624-716639d7b583 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.222674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6c0ad-7111-41d1-b624-716639d7b583 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.223164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6c0ad-7111-41d1-b624-716639d7b583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.243367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:50.321495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c8ef-94d3-4ed4-bb41-5c1c514d459f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:50.349298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6c0ad-7111-41d1-b624-716639d7b583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:50.352012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6c0ad-7111-41d1-b624-716639d7b583 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:50.381173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:50.382478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.382954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.383425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.402579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:50.405216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:50.440564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:50.442124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:50.443232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:50.443232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:50.443232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:50.443232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:50.443232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 371/744] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:50.444795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:50.484833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:50.508903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 374/745] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 13:47:50.511553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c5ac-3184-4962-814f-31ba1c8e20b5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.512036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c5ac-3184-4962-814f-31ba1c8e20b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.512972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c5ac-3184-4962-814f-31ba1c8e20b5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.516060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 375/746] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:50.517939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226c5ac-3184-4962-814f-31ba1c8e20b5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:50.591383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:50.595076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032bc0-f7d4-46eb-85b5-d7ab2a22d990 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:50.652224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:50.653635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.654141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.654631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.675280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:50.678116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:50.679655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226c5ac-3184-4962-814f-31ba1c8e20b5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:50.712382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:50.714138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:50.715882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:50.715882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:50.715882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:50.715882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:50.715882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 372/747] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:50.716427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:50.728610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:50.729286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.729578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.729847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.730566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:50.730873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:50.732266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990721a3-3e58-48c4-a69c-05c3188156de None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:50.749612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 376/748] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:50.752326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.753180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.753969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.783628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:50.785711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:50.860829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1f25-0388-43ee-ab09-a45e0a1c9be6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:50.870993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:50.912967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 377/749] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 13:47:50.914379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:50.932716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:50.934214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:50.935255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:50.936307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:50.956240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:50.958607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:50.989050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:50.990799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:50.993353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:50.993353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:50.993353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:50.993353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:50.997346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 373/750] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:51.000434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:51.009074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:51.094125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b70f6f-873f-4ea2-ba07-2395eaa1ab08 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:51.169677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:51.170839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:51.171351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:51.171846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:51.173288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:51.174103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:51.176567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:51.190244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b1aff-f290-46f4-8b89-1385c725c68b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:51.199266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 378/751] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:51.200897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:51.239678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:51.240952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:51.241501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:51.241979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:51.259165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:51.261954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:51.277263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:51.295288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:51.296857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:51.297992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 374/752] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:47:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:51.300691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:51.361374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a667a9-a68a-4e13-8623-fa0d36af8e27 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:51.418003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:51.419047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:51.419527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:51.420155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:51.421686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:51.424353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:51.426478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:51.456161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 379/753] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:47:51 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:47:51.456161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:51.506981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c3774-61c7-4e64-ae84-e3c2fb0ec021 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:51.572046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:51.573065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:51.573677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:51.574235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:51.575174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:51.575656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:51.577452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:51.596474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 375/754] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:51.597686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:51.643246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:51.650072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c33c5-35af-4dd6-97ff-7cf943d9acfa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:51.729424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:51.731199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:51.731715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:51.732227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:51.752173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:51.755065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:51.783938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:51.785900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 380/755] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:51.788341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:51.819160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a85ec-b824-4b6c-871d-eca7fcd3bbc2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:51.865821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:51.909228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:51.911477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:51.912044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:51.912606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:51.914299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:51.915248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:51.918418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:51.966077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 376/756] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:47:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:51.967705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:52.020771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f0cd6-750d-4d2e-aa6b-8a440a53b313 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:52.094101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:52.096627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.097394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.098232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.114566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:52.118079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:52.147748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:52.149498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:52.151225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:52.155865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 381/757] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:52.158431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:52.207848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:52.212609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d45ab-324f-46eb-9bbe-87ba98640860 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:52.288651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040c467-0321-4040-a786-3df78642e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:52.290316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040c467-0321-4040-a786-3df78642e315 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.291078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040c467-0321-4040-a786-3df78642e315 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.291820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040c467-0321-4040-a786-3df78642e315 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.293559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040c467-0321-4040-a786-3df78642e315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:52.294412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040c467-0321-4040-a786-3df78642e315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:52.298308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040c467-0321-4040-a786-3df78642e315 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:52.323941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 377/758] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:52.325984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040c467-0321-4040-a786-3df78642e315 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:52.398962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a7b65-d393-4cea-8280-08e4f71cd47e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:52.410781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:52.475684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:52.477166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.477671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.479025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.500912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:52.504729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:52.522973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040c467-0321-4040-a786-3df78642e315 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:52.547617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:52.549238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:52.550904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:52.550904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:52.550904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:52.550904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:52.550904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 382/759] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:52.552042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:52.590807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:52.591922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.592461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.592945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.593967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:52.595141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:52.596860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:52.612064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 378/760] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:52.613372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:52.699742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:52.744776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a1bbd-3c74-4257-ba56-487adbdb16c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:52.753743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:52.758732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43722e6f-eb43-46e3-be5a-6028c2a1aa9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:52.804362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:52.805512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.805998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.806459 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.807536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:52.808084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:52.808692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:52.809442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.809791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.810136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.810703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4ba9d-390d-4863-9aa3-99ae72a6778f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:52.811145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=9459ea72747e4ad4a839847d8232bb3c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:52.811532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:52.813651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08e022-d800-4126-ae37-5de19bfa5a51 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:52.820436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 383/761] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:52 2022] GET /identity/v3/role_assignments?user.id=9459ea72747e4ad4a839847d8232bb3c&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:52.828235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46949da-16fa-48c9-8a5a-5a768618f754 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.828235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46949da-16fa-48c9-8a5a-5a768618f754 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.828235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46949da-16fa-48c9-8a5a-5a768618f754 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.828235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 384/762] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:52.832589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46949da-16fa-48c9-8a5a-5a768618f754 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:52.833854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 379/763] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:52.836205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:52.839610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:52.839898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:52.865590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:52.867055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:52.971056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46949da-16fa-48c9-8a5a-5a768618f754 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.006365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 380/764] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:47:53.006365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:53.035590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.036805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.037307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.037859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.039032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=f3eacd1356f448cca079f57f683f37bf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:53.039588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.041661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.050799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:53.062286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 385/765] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:47:52 2022] GET /identity/v3/role_assignments?user.id=f3eacd1356f448cca079f57f683f37bf&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:53.062286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:53.196819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:53.244976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd2d99-6c95-41d7-ae20-9d93462b7df2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.250529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a81ec-d63f-4901-a9b4-9a790886b4ed None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.310412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.312018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.312018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.312018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.320432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.321840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.329663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.331281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.331901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.332538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.333884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.335867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:53.337185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:53.338150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:53.339259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 381/766] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:53.346707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:53.365727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:53.366936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:53.367969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:53.371301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 386/767] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Jul 15 13:47:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:53.378298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:53.517261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:53.573619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0ca33-5f81-4cc5-be4b-e81fbe457fec None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.589392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631825e-7653-4cee-9b01-d530abe5444d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.619908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.620620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.620903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.621180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.621988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:53.622296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.623768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.653740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 382/768] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:53.661447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:53.677400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.678447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.678934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.679413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.680376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:53.680916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.682589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.684698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:53.695211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 387/769] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jul 15 13:47:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:53.704506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:53.807586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999393db-d72e-49b7-8b83-d876e78e8ad0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.847729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8531c-7379-4fbb-aee6-334502dba478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.848865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8531c-7379-4fbb-aee6-334502dba478 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.849143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8531c-7379-4fbb-aee6-334502dba478 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.849427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8531c-7379-4fbb-aee6-334502dba478 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/e424ea6d03414857bc408e49b8df5429/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.851523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8531c-7379-4fbb-aee6-334502dba478 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=e424ea6d03414857bc408e49b8df5429, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.853040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8531c-7379-4fbb-aee6-334502dba478 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.876435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 383/770] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:53 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/e424ea6d03414857bc408e49b8df5429/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:53.876435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a746de-fbbb-4bb9-8d5f-9e1e847f492f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.876435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a746de-fbbb-4bb9-8d5f-9e1e847f492f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.876435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a746de-fbbb-4bb9-8d5f-9e1e847f492f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.877018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 384/771] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:53.879188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.879476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.879802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.893638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe52e1-2179-4e51-8c8e-f1d5c7657e8a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:53.911367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:53.912652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:53.957090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:53.977407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:53.978155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:53.978439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:53.978713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:53.979455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:53.979760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:53.981231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:53.994581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 388/772] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jul 15 13:47:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:54.000824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:54.067618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:54.075686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 385/773] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:47:54.098504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:54.185275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608f3c-061c-44a0-b2e1-5b03646180f1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:54.218924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a938e-0464-4778-a572-9b6494ac0324 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:54.225922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:54.226603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:54.226908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:54.227274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:54.228093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] RBAC: Adding query filter params (interface=public, service_id=8328edd52eda4d6c95d9597a0ec81466, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:54.228446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:54.229998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:54.239312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 389/774] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:47:53 2022] GET /identity/v3/endpoints?service_id=8328edd52eda4d6c95d9597a0ec81466&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:54.249658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:54.257489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:54.258330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:54.258614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:54.258883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] PATH_INFO: `/v3/services/network` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:54.259644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:54.261150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:54.276903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:54.278749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:54.280353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:54.283809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 386/775] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:54 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:54.288577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:54.397263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:54.408985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:54.475222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae68fd-e11f-48be-b617-2e38c009ba2c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:54.553731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:54.554486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56882b-2689-4acf-961c-2f81f623de35 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:54.555155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:54.555759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:54.556711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:54.559615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] RBAC: Adding query filter params (name=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:54.560334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:54.563774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:54.581305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 387/776] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:47:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:54.585498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:54.677749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:54.678354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:54.678648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:54.678929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:54.679646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:54.683475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:54.697546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 390/777] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:47:54 2022] GET /identity/v3/services => generated 772 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:54.783742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6bc3-bf34-435f-86a9-9d7b7b30cd75 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:54.854257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:54.865813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:54.867139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:54.867711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:54.868289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:54.869802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:54.870407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:54.873181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:54.892712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 388/778] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:47:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:54.896947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:55.104813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00fb56-3d4b-45ce-9bae-79eb182a1131 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:55.143077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:55.170010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:55.171164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:55.171653 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:55.172150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:55.174320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] RBAC: Adding query filter params (interface=public, service_id=37e1c3d45a1c468ea18902902cd1bc91, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:55.174888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:55.177662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c674ea-a6a3-4a18-81c3-ad146b667abb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:55.192559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 391/779] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:47:54 2022] GET /identity/v3/endpoints?service_id=37e1c3d45a1c468ea18902902cd1bc91&interface=public®ion_id=RegionOne => generated 194 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:55.198095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:55.368299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd211d-6269-4701-8b61-5db29b5846d8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:55.407419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb266f-a17b-409b-9a29-1c64096b7451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:55.408118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb266f-a17b-409b-9a29-1c64096b7451 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:55.408413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb266f-a17b-409b-9a29-1c64096b7451 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:55.408708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb266f-a17b-409b-9a29-1c64096b7451 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:55.409409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb266f-a17b-409b-9a29-1c64096b7451 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:55.410812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb266f-a17b-409b-9a29-1c64096b7451 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:55.420538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 389/780] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:47:55 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:55.462967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c6034-6d86-4de5-814e-7e69d5f2611c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:55.463640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c6034-6d86-4de5-814e-7e69d5f2611c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:55.464246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c6034-6d86-4de5-814e-7e69d5f2611c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:55.467569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 392/781] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:55.477061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:55.477491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:55.477775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:55.501724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:55.502961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:55.570547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:55.599135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 390/782] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:47:55.654166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c6034-6d86-4de5-814e-7e69d5f2611c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:55.993569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c6034-6d86-4de5-814e-7e69d5f2611c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:56.049770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38038091-be3b-463c-8957-01798d560100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:56.050588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38038091-be3b-463c-8957-01798d560100 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.050869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38038091-be3b-463c-8957-01798d560100 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.051145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38038091-be3b-463c-8957-01798d560100 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.064420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38038091-be3b-463c-8957-01798d560100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-38038091-be3b-463c-8957-01798d560100 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:56.072131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:56.073179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:56.074086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 393/783] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:56.081282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:56.256947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9573e1-83b5-4af4-98d5-7ecc5b9f38be None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:56.287698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9afb4-fbda-452a-b5dd-451a9e00464a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.288050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9afb4-fbda-452a-b5dd-451a9e00464a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.288250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9afb4-fbda-452a-b5dd-451a9e00464a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.292038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 394/784] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:56.305754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.306538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.307424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.321850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:56.322857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.323311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.323735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.324750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:56.325252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:56.326883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:56.339210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:56.340839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:56.346795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 391/785] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:47:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:56.352998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:56.460867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:56.466491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:56.527045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d678-aea1-4ca1-a1e4-667501ad1e6c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:56.532578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 395/786] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:47:56.559550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:56.595449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:56.596595 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.597132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.597620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.622116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:56.624552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:56.651481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:56.652620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:56.653522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:56.653522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:56.653522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:56.653522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:56.665397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 392/787] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:56.670877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:56.770286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96988d38-06ae-4c9e-b95c-583218b4b572 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:56.863905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:56.864758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.867181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.867504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.892668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:56.902184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:56.903174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:56.903992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:56.908987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 396/788] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:47:56.909146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:56.917029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:56.919184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae3fc-00c0-4c96-b92c-70ed7eef6299 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:56.970560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:56.972910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:56.974716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.974716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.975075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:56.976089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:56.976615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:56.979332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdca34-3276-46f5-9f25-0efc6786719c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:56.994551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 393/789] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:56.996323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b73ca-ffa6-4cb1-b7f4-69473d8daad3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:56.996886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b73ca-ffa6-4cb1-b7f4-69473d8daad3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:56.997384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b73ca-ffa6-4cb1-b7f4-69473d8daad3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.003093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 394/790] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:57.004740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b73ca-ffa6-4cb1-b7f4-69473d8daad3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.064436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e7ae3-7f37-4317-ab17-97e910f3200d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.115270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.115885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.116172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.116483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.117575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:57.117891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.119481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9e7bf-bb08-45a0-b956-d93cbff8cf0a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.132340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b73ca-ffa6-4cb1-b7f4-69473d8daad3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.136494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 397/791] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:57.138440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.140128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.140687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.168452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:57.169694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:57.194113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.195424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.196637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.197188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.209342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.211264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:57.233691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:57.235281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:47:57.236238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:57.244315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 395/792] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:57.244315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.262148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 398/793] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:47:57.263471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.269074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:57.300867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:57.388909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21c625-5155-4751-a50e-d15ff84bf621 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.398208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5796e-bf10-4f5f-b43b-855aefeb3d95 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.430886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.431700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.432155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.432467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.446508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.446745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.447201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.447489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.447762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.448224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.448737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:57.448974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.450760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:57.472118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:57.473103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:57.474243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:57.474243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:57.474243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:57.474243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:57.476686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 399/794] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:57.477731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.478693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 396/795] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:57.480420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.576705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:57.597954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:57.620617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83191b76-0f43-48d5-83dc-d51cc26f8281 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.621251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b605cb1-03b8-4caf-a080-4b6d1f4556d1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.669787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.670458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.670669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.670946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.671713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:57.671995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.673505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.673992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.675386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.675881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.676777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.696880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:47:57.697733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 397/796] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:47:57.698785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:47:57.708044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:57.709046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:47:57.709846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:57.718398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 400/797] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:57.718398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.839189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f401555-be8f-405d-b75d-0534194a96ec None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.850447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848466b5-87c0-48ae-81cf-82c58177fbb7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:57.877644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.878417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.878702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.878977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.884608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:57.887846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.889434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.893366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:57.894598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:57.895085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:57.895581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:57.907773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:57.908787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:47:57.909734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:57.911488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:57.913242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:57.913927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 401/798] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:57.914817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:57.919496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:57.934380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:57.935329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:57.936191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:57.936191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:57.936191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:57.936191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:57.939263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 398/799] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:57.940434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.036014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bdcc9-b620-42ba-b0f7-29d3ddcae564 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.052027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1958a87-ff75-4c46-a8e8-3ce7fe8f6f94 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.075635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.076261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.076579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.076854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.077772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:58.078168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.080062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.094870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 402/800] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:58.094870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.098141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.099282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.099771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.100250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.101321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:58.101857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.103506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.124437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 399/801] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jul 15 13:47:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:58.126244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.183812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:58.192026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:58.229909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d30b5b-e46b-47b4-81a2-56ee1d7fe7c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.268257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.268972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.269250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.269521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.270252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:58.270573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.271920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880f52b-d2e2-48ab-9cf6-c4f63798495b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.284796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 403/802] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:58.286010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29342dd-39b7-4f06-a922-4738c48f50fa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.286180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101e865-b539-4280-a463-ed48095f4dba None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.286317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101e865-b539-4280-a463-ed48095f4dba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.286576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101e865-b539-4280-a463-ed48095f4dba None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.287863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 404/803] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:58.293049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101e865-b539-4280-a463-ed48095f4dba None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.328975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.329786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.330073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.330351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.347931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.349818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8eacd34d-bfb3-431f-8650-140d31db8023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:58.370833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:58.371958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:58.373081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:58.373081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:58.373081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:58.373081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:58.373440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 400/804] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:58.376062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86179e-80d7-4f07-99a4-7b990fbb79ee None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.376586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86179e-80d7-4f07-99a4-7b990fbb79ee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.377075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86179e-80d7-4f07-99a4-7b990fbb79ee None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.380176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 401/805] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:47:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:47:58.381237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86179e-80d7-4f07-99a4-7b990fbb79ee None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.411310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101e865-b539-4280-a463-ed48095f4dba None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.450705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:58.460288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.461121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.461395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.461664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.472881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.474620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.493924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6d57bbd7-7d26-4bf6-8b50-2edf2ead353c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:47:58.494257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:58.495107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:58.496056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:47:58.496056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:47:58.496056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:47:58.496056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:58.496056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 405/806] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:58.497106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38813f3-6b31-4261-9d13-f2974a27e650 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.497390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38813f3-6b31-4261-9d13-f2974a27e650 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.497686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38813f3-6b31-4261-9d13-f2974a27e650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.510505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86179e-80d7-4f07-99a4-7b990fbb79ee None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.514967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f38813f3-6b31-4261-9d13-f2974a27e650 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:58.516168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f38813f3-6b31-4261-9d13-f2974a27e650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:58.549765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.550602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.550888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.551167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.558715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.560186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:58.575512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:58.576565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:58.577415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 402/807] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:58.577914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.593886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 406/808] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:47:58.594988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.595337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.595591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.614509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:47:58.615640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:47:58.699690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 407/809] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:47:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:47:58.701208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.712012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299bc9-ef5c-4edc-9197-741d9a3bba87 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.750754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.751427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.751729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.751980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.752739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:58.753052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.754441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.762132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:58.767902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 403/810] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:58.768790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.791428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:58.832261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e372e9-f6aa-4354-a1a0-1f83fb4541f8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.875656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.876548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.876963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.877349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.878171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:58.878541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.880084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.891838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b26d19-d7c6-40b9-9167-0e5345658bdb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:58.892654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 408/811] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:58.893390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:58.933306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:58.933976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:58.934260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:58.934540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:58.935328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:58.935636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:58.937121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:58.950077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 404/812] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:58.950935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.008679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.013668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debfece-1962-4dc7-9e75-7d89f779f9dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.038242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.054535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58587500-2617-491a-b349-b188e7ce181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.055380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58587500-2617-491a-b349-b188e7ce181d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.055662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58587500-2617-491a-b349-b188e7ce181d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.055940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58587500-2617-491a-b349-b188e7ce181d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.056651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58587500-2617-491a-b349-b188e7ce181d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.058081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58587500-2617-491a-b349-b188e7ce181d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-58587500-2617-491a-b349-b188e7ce181d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:59.071025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:59.072953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:59.074207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 409/813] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:59.075100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58587500-2617-491a-b349-b188e7ce181d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.080761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c9845-38e1-43b5-9339-df2d428b7674 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.124471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.125368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.125655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.125934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] PATH_INFO: `/v3/services/network` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.126620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.128070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:59.140631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:59.142896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:59.144715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 405/814] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:58 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:59.146305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.195186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58587500-2617-491a-b349-b188e7ce181d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.232850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.233649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.233926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.234202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.242598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.244062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:47:59.258885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:59.260104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:47:59.262330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:47:59.263344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e55b6b-e17c-43a9-b891-0336f1846b3b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.263488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 410/815] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:59.263488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.289799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.303527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.304335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.304649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.304930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.308286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.313754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.315997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:47:59.331639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:47:59.333097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 406/816] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:47:59.334766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.386267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702a3f7-d99d-44c1-a739-7c9390ab625d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.422640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.423592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.423877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.424155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/9459ea72747e4ad4a839847d8232bb3c/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.440980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=9459ea72747e4ad4a839847d8232bb3c, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.442449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.455937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d055c-9a92-4994-9bde-03b79ab9471a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.457060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 411/817] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:47:59 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/9459ea72747e4ad4a839847d8232bb3c/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:47:59.458449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.501581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.502380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.502678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.502967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.503794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:59.504104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.505666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.517198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 407/818] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:59.517853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.569047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.603656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.606159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d5d89-f7fc-4e2c-9c56-1e4d4a98f331 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.651871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.652585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.652866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.653144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.653875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] RBAC: Adding query filter params (name=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:59.654176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.658580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.664634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 412/819] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:59.665516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.673517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33b521-d641-4259-9ef8-0cd2e9e45744 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.736654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.737782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.738264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.738762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.740352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:59.740946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.743588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.779016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 408/820] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:59.780734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.788975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b254c7-d3a8-434f-b0cb-8cde2a1d6598 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.838496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.839263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.839951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.840497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.841972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:59.842504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.845318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.851256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.871165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 413/821] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:59.872602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:47:59.909741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14ba47-e946-4eb5-a7b1-7fb73d991132 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:47:59.933054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:47:59.955921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:47:59.956634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:47:59.956918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:47:59.957190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:47:59.957917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:47:59.958224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:47:59.960274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:47:59.968456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 409/822] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:47:59.968456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:00.040467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f77f-bb3f-4b22-9f3f-70e471789fca None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:00.079718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:00.080417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:00.080706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:00.081002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:00.081748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] RBAC: Adding query filter params (type=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:00.082060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:00.083445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:00.096538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 414/823] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:00.096538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:00.150345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7935f58-3d2a-415f-9ef6-9e12c30a62f4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:00.180139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:00.221622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:00.222380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:00.222729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:00.223071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:00.244171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddf61c-54dd-4161-b301-70b16f0758db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:00.298473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:00.308981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:00.310560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:00.311040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:00.311519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/f3eacd1356f448cca079f57f683f37bf/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:00.315418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=f3eacd1356f448cca079f57f683f37bf, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:00.318246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:00.326708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:00.329668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:00.348475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 415/824] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:48:00 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/f3eacd1356f448cca079f57f683f37bf/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:48:00.348475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:00.553926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 410/825] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:47:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:48:00.555861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:00.571269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d7887-c2d1-4704-9dc2-64a7e29fd430 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:00.597791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:00.615615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:00.616592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:00.617020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:00.617440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:00.618394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:00.619966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:00.648499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 416/826] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:00.650625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:00.742012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:00.786777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edea3ed-8a77-4edd-ae2e-d6f8b0492283 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:00.870659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b85c-a107-4c18-8586-fd44f15b95c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:00.876241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caf120-9047-4efe-a183-4444510645ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:00.877921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caf120-9047-4efe-a183-4444510645ec None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:00.878740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caf120-9047-4efe-a183-4444510645ec None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:00.879588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caf120-9047-4efe-a183-4444510645ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:00.881457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caf120-9047-4efe-a183-4444510645ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:00.884430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37caf120-9047-4efe-a183-4444510645ec None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:00.930471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 411/827] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:00.968939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:00.971070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:00.972168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:00.974007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:01.000035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:01.004366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:01.043661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:01.046196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:01.048025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:01.048025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:01.048025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:01.048025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:01.052350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 417/828] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:01.059213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caf120-9047-4efe-a183-4444510645ec None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:01.279897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37caf120-9047-4efe-a183-4444510645ec None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:01.344257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:01.345406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:01.346033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:01.346657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:01.349329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:01.349329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:01.355620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:01.358295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:01.404310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 412/829] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:01.414257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:01.456242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:01.583132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0379022-fae9-4e9b-84a7-6008cda5aa2a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:01.655583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:01.657427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:01.658186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:01.658898 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:01.673731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:01.676882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:01.706497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:01.708378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:01.709508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:01.712889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 418/830] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:01.721671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:01.875274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b9a2-53da-4a46-92db-d16dcf15b778 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:01.945172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:01.946371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:01.946864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:01.947365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:01.949005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:01.949562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:01.952487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468aa43-ff29-4830-ada4-62d62b567b8e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:01.978608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 413/831] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:01.988780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:02.069838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5395f-7d8b-4d61-b4a4-738e321e9b5c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:02.070529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5395f-7d8b-4d61-b4a4-738e321e9b5c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:02.071125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5395f-7d8b-4d61-b4a4-738e321e9b5c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:02.080556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 414/832] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:02.083794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:02.084477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:02.085097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:02.115676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:02.117855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:02.155008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:02.197518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868843f-6e0b-4de3-bd08-ba102990d44f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:02.272778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:02.273792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:02.274273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:02.274771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:02.276497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=e424ea6d03414857bc408e49b8df5429) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:02.277058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:02.282307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:02.295239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 419/833] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:01 2022] GET /identity/v3/role_assignments?user.id=e424ea6d03414857bc408e49b8df5429&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:02.402681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:02.444593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 415/834] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:02.481009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:02.696334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6854-d46f-45c5-a308-427804843458 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:02.769059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:02.771161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:02.772048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:02.772881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:02.799074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:02.811951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:02.814419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:02.816081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 420/835] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:02.824444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:03.033921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f322e2-c4f7-4067-ae82-5668ac7cc64a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:03.100496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:03.101663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:03.102239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:03.102729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:03.104138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:03.104735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:03.107334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:03.130924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 416/836] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:03.137544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:03.164471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:03.270216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059299b-3f41-419c-bc0d-313816de718e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:03.311387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:03.312197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:03.312558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:03.312856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:03.324844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:03.326342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:03.348559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:03.351092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:03.353677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:03.353677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:03.353677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:03.353677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:03.353677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 421/837] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:03.364583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:03.458762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:03.561664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5fd39-87a8-4a2c-ac4a-85eab0beb575 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:03.624446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:03.625602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:03.626090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:03.626560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:03.627887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:03.628425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:03.630999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521503a5-fa0e-4d09-b4ad-2b58dfdd5cb8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:03.660548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 417/838] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:03.664055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:03.744831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7587-9606-4cb6-aa50-d9bd69d9f9d0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:03.745323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7587-9606-4cb6-aa50-d9bd69d9f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:03.745806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7587-9606-4cb6-aa50-d9bd69d9f9d0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:03.748485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:03.748736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 418/839] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:03.757307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:03.757778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:03.758226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:03.787115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:03.789171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:03.839274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a2ab-52d2-4e92-a96e-c8fd4da90d7b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:03.889973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:03.892107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:03.892643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:03.893155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:03.907597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:03.911292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:03.922047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 419/840] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:03.945490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:03.948044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:03.949032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:03.950068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:03.950612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 422/841] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:48:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:03.957409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:03.991177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.076567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f7e96-e6e9-4b8f-ac31-fd210993cc80 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.127796 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c94e79-6066-4ead-89b5-b111c0f58b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.129297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c94e79-6066-4ead-89b5-b111c0f58b32 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.130324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c94e79-6066-4ead-89b5-b111c0f58b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.130818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c94e79-6066-4ead-89b5-b111c0f58b32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.134861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94b1e-c516-4f71-b638-1ccc16ba3e11 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.147015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c94e79-6066-4ead-89b5-b111c0f58b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a6c94e79-6066-4ead-89b5-b111c0f58b32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:04.157382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:04.158360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:04.159181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:04.166714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 420/842] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:48:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:04.168257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee716-b111-47c3-ad60-94b70d4f124f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.168578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee716-b111-47c3-ad60-94b70d4f124f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.168715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.169251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee716-b111-47c3-ad60-94b70d4f124f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.170849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 421/843] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:04.171744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee716-b111-47c3-ad60-94b70d4f124f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:04.200591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.201283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.201564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.201839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.202702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:04.203014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:04.205270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1fc6-983f-4ce1-a4eb-e102d0af2bee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:04.228971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 423/844] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jul 15 13:48:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:04.231274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.231774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.232269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.260639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:04.262681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:04.307926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.317393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee716-b111-47c3-ad60-94b70d4f124f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.354825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.355472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.355753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.356030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.356834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:04.357136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:04.358503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:04.374320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 422/845] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:04.374320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:04.389045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 424/846] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:04.389994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:04.519428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed58a1b-e92b-4a98-b85c-9a10823349f5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.523838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.535697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6827bc0-1d90-48b4-b6e0-2f119c00cc1f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.566774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.567576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.567891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.568120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.581658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:04.583120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:04.585503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.586548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.586965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.587390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:04.603718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:04.605588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:04.607354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:04.607354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:04.607354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:04.607354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:04.607354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 425/847] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:04.607354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:04.636907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.689931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:04.691730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:04.745008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064119bb-5c82-460d-afb6-60d8671a06a0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.785786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.786599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.786881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.787264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.787936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:04.789505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:04.803042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:04.805094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:04.806790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 426/848] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:04.808380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:04.818222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 423/849] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:48:04.821025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:04.840906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.896447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:04.945101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3da207-24fb-4ee5-be78-d09ca32973f8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.950893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6ddac-be9f-4d02-9199-b9f018ff8a0f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:04.993068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.993880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.994235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.994581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.995409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:04.995787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:04.996823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:04.997391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:04.997842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:04.998257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:04.998672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:04.999702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:05.000695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.002587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.015449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 424/850] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:05.015449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.021182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 427/851] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:05.026210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.131574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:05.150958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127aad4-3361-4913-a4a1-8162d26bbfd5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.168150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4da14d-de14-4559-9c77-eae83c698ef1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.178290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:05.196373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.198313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.198869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.199335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.213060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.213807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.214024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.214299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.215029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:05.215332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.216768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.219413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.221164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.222746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 428/852] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:05.224138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:05.265536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:05.267115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:05.269169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:05.269169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:05.269169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:05.269169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:05.272377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 425/853] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:05.280257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.417096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7720-d634-486b-9cb0-5ea5829f403d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.470341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.471079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.471359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.471642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.472640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cde8f-662b-4432-9cb7-953247c16d95 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.481182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.482673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-36ad3b69-5323-4062-ab32-d9a56ebad81d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:05.499704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:05.500796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:05.501718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 429/854] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:05.503030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eea05-84d3-4296-b589-ea216d4ada17 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.503030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eea05-84d3-4296-b589-ea216d4ada17 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.503030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eea05-84d3-4296-b589-ea216d4ada17 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.504750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 430/855] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:05.510554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.511118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.511118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.520376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:05.533056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:05.534290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:05.537326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.539128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.539417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.539688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.540485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:05.540803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.542328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.568694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 426/856] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:05.568694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.612040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:05.632582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 431/857] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:05.633585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.702854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc18f00-6d62-4a8a-ab58-54d5ed85ffc6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.745993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.746713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.746889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.747150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.748009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:05.748321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.749845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.772714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 427/858] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:05.773473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.811124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db07d-1f3f-4427-9b2b-f8ff6aac8ffb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.858397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.859205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.859538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.859812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.862073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:05.873086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.874553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:05.889144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:05.890230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:05.891671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 432/859] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:05.892350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:05.917413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:05.929544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b237e5-0d9f-419f-803a-f9984df1b8e0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:05.972763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:05.973935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:05.974155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:05.974155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:05.975021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:05.976431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:05.989116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:05.990230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:05.991057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 428/860] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:05.991964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.028002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909ea1c-7c31-4e2f-86b2-90a2c9abcdaf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.070160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.071183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.071658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.072966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.115512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62df878-b361-4de4-9402-7f6a2c874bf0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.155777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:06.166993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.169194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.186550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.189044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.189661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.190095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.191389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:06.191853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.193976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.211948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 429/861] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:06.213150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.238286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:06.311557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 433/862] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:48:06.312862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.370985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472fbca-7aaf-49a1-a397-45c4bedb4717 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.419421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.420091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.420372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.420671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.421465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:06.421681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.423649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.434283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 430/863] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:06.435212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.457895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:06.472314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0434c4b-7f47-47c3-9f2c-f6f08731510a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.524511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.525182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.525461 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.525739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.526578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=9459ea72747e4ad4a839847d8232bb3c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:06.526881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.528727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.536810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:06.537875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 434/864] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:06 2022] GET /identity/v3/role_assignments?user.id=9459ea72747e4ad4a839847d8232bb3c&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:06.538549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.588619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a78d-2a14-4683-bed2-ed009cd77bcd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.640502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.642706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.643225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.643857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.665906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.667769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:06.696538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:06.697478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:06.698350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:06.698350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:06.698350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:06.698350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:06.700697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 431/865] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:06.706431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fceb2-8a90-4388-8d8d-96573223c47b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.707020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.756200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.757073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.757451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.757828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.758692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:06.758992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.760380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.767307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 435/866] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:06.864477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac571d4-d0a5-46fa-9e58-805d6232f899 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:06.907227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:06.913990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:06.915957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:06.917237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:06.917895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:06.919444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:06.920073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:06.922487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:06.937375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 432/867] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:06.947045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:06.996631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:07.096189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977927e6-0748-4964-b327-53a6cda4200d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:07.136614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:07.137762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:07.138265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:07.138773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:07.147923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:07.149812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:07.166173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:07.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:07.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:07.171154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 436/868] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:07.179342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:07.316555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708fe0f-b11d-405f-8af4-2b3cc4683a73 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:07.355389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:07.356537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:07.357036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:07.357539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:07.358581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:07.359105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:07.361414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7598e-462a-4a14-8f4d-319af9e5106e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:07.376609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 433/869] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:07.387111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:07.454432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802600-cc49-46ce-9e5f-19d3605615d7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:07.454773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802600-cc49-46ce-9e5f-19d3605615d7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:07.455031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802600-cc49-46ce-9e5f-19d3605615d7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:07.456610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 434/870] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:07.463754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665bf3f-3fab-4cf4-8d3e-a879d34e2169 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:07.464443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665bf3f-3fab-4cf4-8d3e-a879d34e2169 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:07.465152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665bf3f-3fab-4cf4-8d3e-a879d34e2169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:07.486672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6665bf3f-3fab-4cf4-8d3e-a879d34e2169 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:07.488380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6665bf3f-3fab-4cf4-8d3e-a879d34e2169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:07.501756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:07.531330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b55d9f-4e61-44f3-9997-6e95ada31364 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:07.567896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:07.573548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:07.574653 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:07.575213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:07.575726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:07.576859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=f3eacd1356f448cca079f57f683f37bf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:07.577426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:07.579194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:07.586763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 437/871] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:07 2022] GET /identity/v3/role_assignments?user.id=f3eacd1356f448cca079f57f683f37bf&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:07.599852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 435/872] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:07.623589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:07.748633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09302fc-cb5b-4dc3-bf6e-7ebc2ad3006f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:07.791947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a502c-ea8f-4527-8e6d-5bdef48b7d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:07.792673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a502c-ea8f-4527-8e6d-5bdef48b7d81 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:07.792957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a502c-ea8f-4527-8e6d-5bdef48b7d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:07.793231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a502c-ea8f-4527-8e6d-5bdef48b7d81 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:07.794030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a502c-ea8f-4527-8e6d-5bdef48b7d81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:07.795459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a502c-ea8f-4527-8e6d-5bdef48b7d81 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:07.814693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 438/873] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:48:07 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:08.818422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463073d-83d4-4c51-95ca-cf44c754f8ad None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:08.818991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463073d-83d4-4c51-95ca-cf44c754f8ad None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:08.818991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463073d-83d4-4c51-95ca-cf44c754f8ad None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:08.820965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 436/874] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:08.826971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac39c4-d797-4431-b867-725a38b1ac1f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:08.827529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac39c4-d797-4431-b867-725a38b1ac1f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:08.827529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac39c4-d797-4431-b867-725a38b1ac1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:08.839706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:08.845791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0eac39c4-d797-4431-b867-725a38b1ac1f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:08.847918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0eac39c4-d797-4431-b867-725a38b1ac1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:08.935729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 439/875] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:08 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:08.959067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463073d-83d4-4c51-95ca-cf44c754f8ad None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.050139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f980105-e983-40d0-bbd9-863e1907ce79 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.050495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f980105-e983-40d0-bbd9-863e1907ce79 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.050700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f980105-e983-40d0-bbd9-863e1907ce79 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.052402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 440/876] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:09.058013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.058311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.058595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.076136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:09.077440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:09.084514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463073d-83d4-4c51-95ca-cf44c754f8ad None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:09.132733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:09.133569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.134461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.134756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.149645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:09.158450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:09.159398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:09.160214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 437/878] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jul 15 13:48:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:09.160959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 441/878] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:09.169488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.181598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.225926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:09.253426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:09.304997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a148f4-7391-449b-a5e5-66480986657c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:09.309114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877b82d-732e-4959-9c22-39e82b60fd1f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:09.342584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:09.343355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.344278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.344694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.345586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:09.345998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:09.347545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:09.356531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:09.357354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.357741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.358122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.358963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:09.364531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:09.364739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 438/879] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:48:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:09.367382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.387165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 442/880] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:48:09 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:48:09.490763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3443-80b2-46dc-b467-e789b2d4288e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:09.493632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:09.537548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:09.538357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.538641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.538916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.551800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:09.553395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:09.571939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:09.572968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:09.573886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:09.573886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:09.573886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:09.573886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:09.580276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 439/881] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:09.581099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.695771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e2587-efc3-44c3-8814-4509a6ef6453 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:09.721722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:09.736495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:09.737176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.737456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.737732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.738465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:09.738765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:09.740155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2698-da92-4b99-af2b-a006c3a2401b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:09.755639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 443/882] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:09.761405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.818935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32071e-a15a-47fd-be4c-d401896bc357 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.819285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32071e-a15a-47fd-be4c-d401896bc357 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.819500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32071e-a15a-47fd-be4c-d401896bc357 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.821239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 444/883] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:09.826885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.827203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.827732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.845999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:09.847235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:09.897818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f0c1-7b45-4996-a9f2-39008da8ef18 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:09.932675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 445/884] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:09.944543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:09.945762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:09.946311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:09.946809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:09.948882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:09.958002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:09.959480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:09.981069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:09.982086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:09.983045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:09.983593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 440/885] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:48:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:09.992503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:10.032810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:10.090448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd73b-a950-4342-ae50-0c922cb96f1b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.108725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:10.136932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:10.138673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.139167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.139662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.141485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:10.149913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:10.168112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:10.169115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:10.169935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:10.176860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 446/886] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:10.177954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:10.198147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6fc9f-ab5f-44e3-83b9-bc56eb1b8334 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.262815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:10.264003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.264541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.264999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.266588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:10.267134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:10.269834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac249d5-ce4a-47d2-991b-d4fc7d0ece58 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:10.294280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 441/887] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jul 15 13:48:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:10.296537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b0f1c-73d9-4ec0-bf23-4adf5bee9b11 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.297601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b0f1c-73d9-4ec0-bf23-4adf5bee9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.298087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b0f1c-73d9-4ec0-bf23-4adf5bee9b11 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.300600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 442/888] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:10.305328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b0f1c-73d9-4ec0-bf23-4adf5bee9b11 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:10.305597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130c269-9e61-446e-890a-c9acc32ee415 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.344575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:10.345724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.346234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.346619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.347444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:10.347843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:10.349755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf68f-7ace-4f16-94c8-4382fe5b477f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:10.360136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 447/889] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:10.363201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.363673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.364083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.389999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:10.392422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:10.404357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:10.448794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:10.457551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b0f1c-73d9-4ec0-bf23-4adf5bee9b11 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.516543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 448/890] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:10.516543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:10.518520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:10.519281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.519645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.520007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.611972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:10.613599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:10.650723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad038f-10bf-4b83-9261-d7e680da7bab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.690618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:10.691391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.691729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.692065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.692826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:10.693127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:10.694511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:10.697222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:10.701366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 449/891] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:10.702027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:10.764863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:10.776818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 443/892] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:10 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:10.784078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:10.913881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ece1c-8ca6-4da8-b3cd-738eb5e75d8c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.932154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617544b-d76e-4128-8c19-01f512f4d27b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:10.951810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:10.952596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:10.952942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:10.953713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:10.965879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:10.967447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:10.988937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:10.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:10.992512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:10.992512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:10.992512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:10.992512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:10.994953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 444/893] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:10.996296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.011326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.013215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.013976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.014779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.016987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.019928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.024541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:11.049569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:11.051545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:11.053234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:11.058753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 450/894] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:11.066567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.107958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99aeec-62c7-41bc-9ec4-122b63be1d65 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.147667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.147667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.147667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.147667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.148114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:11.148185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.149690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.167396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 445/895] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:11.168735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.264674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:11.284116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:11.286995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc21ee-a5f2-4362-8067-385677ef577f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.292551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57671d-d242-46fd-a404-dd3ba2435f21 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.325597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.326437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.326717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.327013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.335592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.337761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:11.357396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:11.359735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:11.361555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:11.364221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 446/896] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:11.365939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.370754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.372489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.373341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.374114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.376611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:11.377747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.381043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.392360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 451/897] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:11.404429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.488981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e7ce-2bee-4772-8d62-e49cd1ef429c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.523921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d475-33d6-4252-b3db-5f0af119c0fc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.539856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.540576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.540868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.541320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.542604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:11.542921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.544741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.561910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 447/898] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:11.568311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.571589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.572063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:11.572276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.572594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.572875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.573607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:11.574027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.576051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.586524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 452/899] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:11.587953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.620446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:11.696645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a29fa9-9a03-415b-8e65-500337e57a7d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.703938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b5aad-5b57-4c70-82c5-4f183957dc27 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.736190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.737193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.737682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.738140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.739185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] RBAC: Adding query filter params (role.id=c03bc669e9e046598e003e38eb44dc89, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=9459ea72747e4ad4a839847d8232bb3c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:11.739726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.741818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.743364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.744042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.744324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.744706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.745519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] RBAC: Adding query filter params (interface=public, service_id=3ce9dd46d5514db084fe5ce23ff657d7, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:11.745833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.747228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf85035-a11d-4e84-9d4f-dd3f037f6ab1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.754344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 448/901] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/role_assignments?user.id=9459ea72747e4ad4a839847d8232bb3c&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=c03bc669e9e046598e003e38eb44dc89 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:11.757921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 453/901] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/endpoints?service_id=3ce9dd46d5514db084fe5ce23ff657d7&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:11.759186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:11.845963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:11.894462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26298026-3ec7-4686-a731-f48698cf673a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:11.936267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16057d9-fe67-4be9-b402-38860d57f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:11.937775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16057d9-fe67-4be9-b402-38860d57f69a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:11.938210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16057d9-fe67-4be9-b402-38860d57f69a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:11.938819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16057d9-fe67-4be9-b402-38860d57f69a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:11.939681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16057d9-fe67-4be9-b402-38860d57f69a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:11.941266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16057d9-fe67-4be9-b402-38860d57f69a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:11.948476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 449/902] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:48:11 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 13:48:12.240816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e19caf-e9d7-46dd-a7da-6e1c7085e5a2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:12.241392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e19caf-e9d7-46dd-a7da-6e1c7085e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:12.241794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e19caf-e9d7-46dd-a7da-6e1c7085e5a2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:12.244869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 454/903] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:12.254389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:12.254885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:12.255382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:12.282728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:12.284362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:12.362954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 450/904] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:12.401943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e19caf-e9d7-46dd-a7da-6e1c7085e5a2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:12.535674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:12.555438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e19caf-e9d7-46dd-a7da-6e1c7085e5a2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:12.596278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:12.597263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:12.597563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:12.597845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:12.598808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:12.600296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:12.614497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:12.615533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:12.616714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 455/905] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Jul 15 13:48:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:12.627237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:12.739411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:12.823197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77794cde-302a-4f27-aaa1-e8aa13e0c934 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:12.897816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:12.899105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:12.899685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:12.900178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:12.901493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:12.902037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:12.904737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:12.916329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 451/906] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:12.925961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:13.054274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107ca18-20a2-4b6e-83f9-4ad4a672900e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:13.092753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:13.093355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.094190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.094471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:13.095212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:13.095522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:13.097005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:13.104065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 456/907] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:13.113247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:13.282855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5431ca5-96be-48aa-93cd-b4ee9ea936ee None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:13.333218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:13.361999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:13.363028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.363504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.363976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:13.365508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] RBAC: Adding query filter params (interface=public, service_id=1698b691c78b4aa79c9b82a17c55400d, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:13.366049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:13.368678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe1c1-e0ea-45eb-8b31-243cdeb9d176 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:13.384240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 452/908] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:48:13 2022] GET /identity/v3/endpoints?service_id=1698b691c78b4aa79c9b82a17c55400d&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:13.393698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:13.428759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:13.555040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f93fd-28d1-46cb-94fc-cb8fdd84df2e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:13.592867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3960ce6-6e92-4cba-ba72-8b7c9374f2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:13.594059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3960ce6-6e92-4cba-ba72-8b7c9374f2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.594349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3960ce6-6e92-4cba-ba72-8b7c9374f2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.594633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3960ce6-6e92-4cba-ba72-8b7c9374f2a0 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:13.595351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3960ce6-6e92-4cba-ba72-8b7c9374f2a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:13.596813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3960ce6-6e92-4cba-ba72-8b7c9374f2a0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:13.604041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 457/909] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:48:13 2022] GET /identity/v3/services => generated 1268 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 13:48:13.902189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff4dff-3688-4c70-9e5b-6f21646cb57b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.902695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff4dff-3688-4c70-9e5b-6f21646cb57b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.903178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff4dff-3688-4c70-9e5b-6f21646cb57b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:13.906702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 453/910] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:13.915855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.916360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.919040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:13.961508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:13.963713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:13.981500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe414004-6e01-4498-b272-920d89c29f5d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.982152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe414004-6e01-4498-b272-920d89c29f5d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.982631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe414004-6e01-4498-b272-920d89c29f5d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:13.986411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 454/911] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:13.991601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:13.992242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:13.992757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:14.011653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:14.013378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:14.088412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 458/912] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:14.107575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:14.141012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 455/913] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:14.142591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:14.187876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:14.259861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff597cf6-b81b-43b3-8dfc-3498cc2dc5c9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:14.306508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:14.307650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:14.308164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:14.308864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:14.309827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:14.312170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:14.340620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 459/914] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:48:14 2022] POST /identity/v3/services => generated 241 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:48:14.383743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8ed24-febf-448c-bd18-f59d694c5d48 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:14.409007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:14.439359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b058a9c-ebaa-41a4-b6fc-9575b6052c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:14.440615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058a9c-ebaa-41a4-b6fc-9575b6052c99 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:14.441080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058a9c-ebaa-41a4-b6fc-9575b6052c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:14.441530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058a9c-ebaa-41a4-b6fc-9575b6052c99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:14.455636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b058a9c-ebaa-41a4-b6fc-9575b6052c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2b058a9c-ebaa-41a4-b6fc-9575b6052c99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:14.464472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:14.465477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:14.466332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:14.469082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 456/915] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jul 15 13:48:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:14.478920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:14.622195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:14.681733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f6fe-37ca-4457-a2f2-1e5bb20f401d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:14.682149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd46f6-a519-41ba-9948-b08cebb8876c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:14.682607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd46f6-a519-41ba-9948-b08cebb8876c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:14.683066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd46f6-a519-41ba-9948-b08cebb8876c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:14.685732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 457/916] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:14.693956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:14.694164 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:14.694521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:14.716237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:14.717584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:14.746900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:14.748722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:14.749493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:14.750797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:14.752776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:14.753601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:14.756804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:14.774874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 460/917] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:48:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:14.784944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:14.799945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 458/918] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:14.840357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:14.988138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0eaae-7764-41c9-8dce-7c7aaeda0387 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.021103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdc772-2e17-452b-be84-aaeca8e99fa7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.025474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:15.058189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.059865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.060586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.061235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.065684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:15.082719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.084130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.084309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.084837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.085416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.086635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.087779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.089483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:15.112750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:15.115042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 13:48:15.116994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:15.125502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 459/919] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:15.128980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:15.130939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:15.133110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:15.133110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:15.133110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:15.133110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:15.133613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.140529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 461/920] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:15.149839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.298972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2b78a-bab5-4613-a8bb-d8bb8a4b4622 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.350090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bcf27-ea8e-4ba8-97ce-d89d4e26d08d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.354550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.355972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.356722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.357434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.360434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:15.361327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.365745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.381898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 460/921] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:15.390324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.399139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.399821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.400097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.400369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.401206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:15.401514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.402899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.424424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 462/922] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:15.429706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.445461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:15.523266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:15.548520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c4a69-dceb-4aac-9eb2-231aeb05b17a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.607054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076575d9-f4a4-4946-a677-9a0330f8723e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.613543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.615098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.615969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.616760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.618412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:15.619223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.622049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.639356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 461/923] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jul 15 13:48:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:15.641861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.645964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.646786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.647062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.647333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.657520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.658989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:15.680701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:15.682503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:15.684331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:15.686942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 463/924] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:48:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:15.688862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.823737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1f8a-991d-4513-bc88-c0bed42d9808 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.833219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:15.833818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:15.846809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af354d7-ef13-4a21-8db5-4c1ebf76d1f9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:15.868350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.869600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.870088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.870737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.872420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:15.873127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.876120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c59146-102f-4c9f-942d-2426c30a4f30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.891428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6fd4e-5d63-49e1-a792-f55833dca88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:15.892101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6fd4e-5d63-49e1-a792-f55833dca88d None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.892412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6fd4e-5d63-49e1-a792-f55833dca88d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.892698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6fd4e-5d63-49e1-a792-f55833dca88d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.893474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6fd4e-5d63-49e1-a792-f55833dca88d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:15.894902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6fd4e-5d63-49e1-a792-f55833dca88d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:15.914694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 464/925] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jul 15 13:48:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:15.917405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6b96c-8bb2-446b-b2d9-418b749d54d5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.917920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6b96c-8bb2-446b-b2d9-418b749d54d5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.918431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6b96c-8bb2-446b-b2d9-418b749d54d5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.928416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 465/926] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:15.928416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6b96c-8bb2-446b-b2d9-418b749d54d5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:15.929151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 462/927] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:15.930825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:15.931295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:15.931728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:15.951604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:15.953972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:16.123294 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 463/928] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:16.141839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6b96c-8bb2-446b-b2d9-418b749d54d5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:16.165848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:16.209159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:16.210611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:16.211114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:16.211798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:16.240677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:16.244112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:16.280944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:16.281872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:16.282713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:16.282713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:16.282713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:16.282713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:16.282937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:16.285760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 466/929] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:16.298257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:16.318102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f123ba6-f004-4f4e-b438-a799fc3614a0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:16.336374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:16.357915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:16.359741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:16.360226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:16.360736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:16.361615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:16.363225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:16.378881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:16.379810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 13:48:16.380646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:16.383747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 464/930] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Jul 15 13:48:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:16.394884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:16.521701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe49c4-2e30-4b14-864b-34410821f244 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:16.592676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5bc68-8466-4330-910e-bbab77e7b29e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:16.606198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:16.607054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:16.607197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:16.607529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:16.608239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:16.608523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:16.611423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:16.632364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 467/931] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:16.633385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:16.691671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:16.693369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:16.694451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:16.695362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:16.697220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:16.698475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:16.701784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:16.719378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 465/932] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:16.725426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:16.744649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:16.753996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:16.797394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f7143-3500-4e66-9c50-38940836fbc6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:16.874268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:16.877108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:16.877974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:16.879105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:16.897850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:16.902198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:16.942809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:16.944681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:16.946538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:16.948795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 468/933] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:16.954920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:16.980314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e9d5b-770c-40d0-9fd5-2760513a2af0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:17.072798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:17.074457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.075245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.076032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.078233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:17.078430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abf0ad-e2a8-46f2-a2eb-a645eed0347f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:17.079536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:17.082491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c785-565e-4acc-bdfd-6ec0e65c0bd8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:17.099783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 466/934] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:17.111141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69afce-7875-44c8-9111-aa1736bf970d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.112083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69afce-7875-44c8-9111-aa1736bf970d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.112868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69afce-7875-44c8-9111-aa1736bf970d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.115745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 467/935] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:17 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:17.117520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69afce-7875-44c8-9111-aa1736bf970d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:17.124252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:17.125343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.125625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.125897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.126699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:17.127008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:17.128496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97a44f-cf59-49a3-88cb-9a4a9822f368 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:17.131937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:17.149270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 469/936] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:17.150706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.150994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.151275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.172433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:17.173719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:17.250781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:17.264475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 470/937] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:17.264730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:17.334452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69afce-7875-44c8-9111-aa1736bf970d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:17.396724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:17.397895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.398192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.398474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.399248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:17.401281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:17.428604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 468/938] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:17.430642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:17.462139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fc770-cee1-45d2-8ea7-717d5aa3563e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:17.511174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:17.536832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163135-f460-437b-84d8-264b513c281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:17.537952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163135-f460-437b-84d8-264b513c281f None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.538310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163135-f460-437b-84d8-264b513c281f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.538661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163135-f460-437b-84d8-264b513c281f None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/9459ea72747e4ad4a839847d8232bb3c/roles/c03bc669e9e046598e003e38eb44dc89` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.542284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163135-f460-437b-84d8-264b513c281f None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=9459ea72747e4ad4a839847d8232bb3c, role_id=c03bc669e9e046598e003e38eb44dc89)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:17.544009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163135-f460-437b-84d8-264b513c281f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:17.563908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 471/939] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:48:17 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/9459ea72747e4ad4a839847d8232bb3c/roles/c03bc669e9e046598e003e38eb44dc89 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:48:17.616865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfa975-e666-4319-969c-b5485f71a985 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:17.649637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:17.665568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:17.667489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.668053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.668446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.669527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:17.670993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:17.685671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:17.687736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:17.688969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 469/940] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:17.693572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163135-f460-437b-84d8-264b513c281f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:17.833433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163135-f460-437b-84d8-264b513c281f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:17.875893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:17.876970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:17.877494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:17.877978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:17.878910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:17.879434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:17.881082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:17.887317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 472/941] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:17.897447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:18.072864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2060a-6e7e-410e-9404-3f42b6a253d9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:18.133679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:18.134625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:18.134908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:18.135228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:18.136258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:18.136612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:18.138310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:18.146002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 470/942] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:18.151326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:18.213864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:18.276858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae661895-e4df-4cc2-abbe-ac714b21cde9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:18.315097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:18.315770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:18.316109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:18.316313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:18.317155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] RBAC: Adding query filter params (interface=public, service_id=fed71697818b4a4fb551921f3a27bd6d, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:18.317465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:18.318860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a9786-2f0c-44bb-99c7-b9aa79642e99 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:18.325273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 473/943] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:48:18 2022] GET /identity/v3/endpoints?service_id=fed71697818b4a4fb551921f3a27bd6d&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:18.331655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:18.412281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:18.478417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff684d24-e124-4fd5-859b-88f40381b128 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:18.517422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d31f5a-f1e7-4c91-8acb-e28335216393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:18.518105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d31f5a-f1e7-4c91-8acb-e28335216393 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:18.518388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d31f5a-f1e7-4c91-8acb-e28335216393 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:18.518667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d31f5a-f1e7-4c91-8acb-e28335216393 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:18.519359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d31f5a-f1e7-4c91-8acb-e28335216393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:18.520774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d31f5a-f1e7-4c91-8acb-e28335216393 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:18.527742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 471/944] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:48:18 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 13:48:18.670664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cc61b-80b8-4ba9-8b48-fa4a4680ac4d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:18.670852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cc61b-80b8-4ba9-8b48-fa4a4680ac4d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:18.671137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cc61b-80b8-4ba9-8b48-fa4a4680ac4d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:18.673530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 474/945] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:18.679262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:18.679619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:18.679820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:18.698542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:18.699778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:18.777146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 472/946] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:18.815586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cc61b-80b8-4ba9-8b48-fa4a4680ac4d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:18.970418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cc61b-80b8-4ba9-8b48-fa4a4680ac4d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:18.989840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:19.025306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:19.026074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.026433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.026857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.027575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:19.029126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:19.031331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING keystone.server.flask.common [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 13:48:19.050020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 13:48:19.050020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 13:48:19.050020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [parameters: {'id': '8fc2895afa6c46e1b48d2b7fe98ed504', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 13:48:19.050020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8fc2895afa6c46e1b48d2b7fe98ed504.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8fc2895afa6c46e1b48d2b7fe98ed504. Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 13:48:19.057960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 15 13:48:19.058752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jul 15 13:48:19.059689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 13:48:19.060583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 13:48:19.061467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 13:48:19.062275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'id': '8fc2895afa6c46e1b48d2b7fe98ed504', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:19.063104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:19.063856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8fc2895afa6c46e1b48d2b7fe98ed504. Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:19.068928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 475/947] 10.222.0.16 () {60 vars in 1140 bytes} [Fri Jul 15 13:48:18 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:19.069481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:19.137888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:19.228003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d37589-0114-4236-b35b-2ad72c830842 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:19.266126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:19.266940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.267223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.267494 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.275236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:19.276730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9aeacdaf-6b4c-460f-a0ad-1ab4d8a0e4ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:19.292157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:19.295285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:19.297988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 473/948] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:19.300449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:19.465805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c13364-4f9b-4316-a6c6-e687cbbdf357 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:19.517090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:19.518148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.518644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.519144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.520604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:19.521158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:19.523323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb563-57a8-499f-8684-50094ad8016d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:19.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 476/949] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:19.801899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56cd2-0ad7-4910-bd9f-26f3558c80da None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.802666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56cd2-0ad7-4910-bd9f-26f3558c80da None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.802666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56cd2-0ad7-4910-bd9f-26f3558c80da None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.803850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 474/950] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:19.809968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.810388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.811051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.812432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eba90-7cf3-4c17-87a3-4b8fb3eb4c1b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.812728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eba90-7cf3-4c17-87a3-4b8fb3eb4c1b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.813012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eba90-7cf3-4c17-87a3-4b8fb3eb4c1b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.814679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 475/951] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:19.820040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:19.820327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:19.820654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:19.830330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:19.831893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:19.838671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:19.839874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:19.861498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:19.932753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 477/952] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:19.955836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:19.976419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 476/953] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:20.005357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:20.064370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:20.145484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c927bdb-f071-4b22-86ae-fb4ded0ab379 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.153126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d81c-f838-4849-a1b5-13a6765c31fe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.198228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.199501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.199979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.200828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.202250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:20.204872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:20.207972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.208966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.209294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.210133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:20.223961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:20.225210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:20.226313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:20.227790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:20.231800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 477/954] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:20.234795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:20.236882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:20.238466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:20.246545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:20.246811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 478/955] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jul 15 13:48:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:20.247978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:20.276830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:20.391296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd57a5-b1c9-4971-b4b4-9c79ed3b9ea4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.393373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:20.413091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321b582-2aaa-46dc-afd7-6d7d3a52ac8f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.434190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.435134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.435708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.436147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.437742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:20.438247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:20.439891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:20.446390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 478/956] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:20.455415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:20.479849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.480639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.481249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.481249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.482712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:20.483260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:20.486389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:20.501024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 479/957] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:20.507562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:20.593683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476f465-8bb6-4d69-ab4b-98de7c8beee6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.617217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:20.643446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.644040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.644319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.644622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.645343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:20.645591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00473296-eb1c-4960-8bf1-87e253513c83 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.645714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:20.647014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:20.660569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 479/958] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:20.690825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.691651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.691931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.692203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.705621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:20.707175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-675fb1c1-419d-4b6b-b4ce-0bb4e3d2653e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:20.727582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:20.728570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:20.729481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:20.729481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:20.729481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:20.729481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:20.729481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 480/959] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:20.735242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:20.775707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:20.877244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e96b4-ea94-46dd-af1f-261e101830a7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:20.944226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:20.945346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.945821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.946323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.947692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:20.948260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:20.950714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:20.964419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e3723-d4e1-4823-a20c-3f7ca08d287d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.965213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e3723-d4e1-4823-a20c-3f7ca08d287d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.965213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e3723-d4e1-4823-a20c-3f7ca08d287d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.967069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 481/960] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:20.972751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:20.973406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:20.973811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:20.976748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 480/961] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:20.985255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.004439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:21.004439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:21.101883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 482/962] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:21.126086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.142063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:21.155431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06aa6f-9d55-4cfb-9129-6ba7e115165e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.204414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.205622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.206146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.206638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.215768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.217660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:21.236549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:21.237548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 13:48:21.238510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:21.245908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 481/963] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:48:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:21.247253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.253662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:21.325531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da61f8-210f-46e6-b9b6-325d62577b60 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.365470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.366650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.367100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.367612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.368482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.370053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:21.384113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:21.385261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:21.386192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:21.386940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7338f14-da50-4b21-b413-c9e8fc40e316 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.389033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 483/964] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:21.394835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.438982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.440219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.440701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.441124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.442568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:21.444054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.445852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:21.472086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 482/965] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jul 15 13:48:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:21.478081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.494760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:21.533251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e395345-1b65-4640-be53-b3a0274b4360 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.561388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:21.578535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.580231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.581270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.582309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.583974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:21.584930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.588055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:21.599812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038da7e-614a-47c4-9822-778fc4ecb2dc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.600615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 484/966] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:21.607322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.646181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63c33c-658c-44dd-9cc3-884dd9f51084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.646860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63c33c-658c-44dd-9cc3-884dd9f51084 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.647201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63c33c-658c-44dd-9cc3-884dd9f51084 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.648365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63c33c-658c-44dd-9cc3-884dd9f51084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.730682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63c33c-658c-44dd-9cc3-884dd9f51084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.732098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63c33c-658c-44dd-9cc3-884dd9f51084 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:21.743835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:21.745904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f05055-ace4-4c2b-ac92-5549da0579fe None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.789249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.790339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.790949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.791444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.792527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:21.793085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.794781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:21.801508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 485/967] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:21.808485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.863478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 483/968] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:48:21.865044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31516bf-78d8-4e8e-82c7-6b6fe2715bbc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.865361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31516bf-78d8-4e8e-82c7-6b6fe2715bbc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.865621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31516bf-78d8-4e8e-82c7-6b6fe2715bbc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.867252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 484/969] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:21.869433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31516bf-78d8-4e8e-82c7-6b6fe2715bbc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:21.890617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:21.945392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea32621-ec9f-45c4-9cc2-996963f73505 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.992918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d77f-7a3b-4f9f-9546-6ba6a273e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:21.993944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d77f-7a3b-4f9f-9546-6ba6a273e2bc None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:21.993944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d77f-7a3b-4f9f-9546-6ba6a273e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:21.994229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d77f-7a3b-4f9f-9546-6ba6a273e2bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:21.994788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38d77f-7a3b-4f9f-9546-6ba6a273e2bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:21.995136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31516bf-78d8-4e8e-82c7-6b6fe2715bbc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:21.996197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38d77f-7a3b-4f9f-9546-6ba6a273e2bc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:22.007556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:22.023829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 486/970] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:22.025276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.025585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.025919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.058406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:22.060484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:22.071332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:22.073224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.074154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.074890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.096125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:22.099910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:22.151671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:22.153745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:22.156225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:22.156225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:22.156225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:22.156225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:22.157100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 485/971] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:22.168340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:22.213179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 487/972] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:22.240269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:22.307699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:22.348606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f4fad-a964-42b7-8e58-a11663486006 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:22.392152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:22.393341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.393866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.394355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.395327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:22.395846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:22.397842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:22.418020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 486/973] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:22.422642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:22.441746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8117b-2989-4d67-bf70-71653d5a7089 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:22.450761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:22.511729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:22.512979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.513487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.513992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.515593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:22.518282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:22.542075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 13:48:22.542075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 13:48:22.542075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [parameters: {'user_id': '44b949fa6be446a8985584f397dc86cc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 13:48:22.542075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 15 13:48:22.560853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 13:48:22.563594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 13:48:22.566308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 13:48:22.568085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 13:48:22.572163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 13:48:22.575818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'user_id': '44b949fa6be446a8985584f397dc86cc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:22.580516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 15 13:48:22.583529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 488/974] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jul 15 13:48:22 2022] POST /identity/v3/users => generated 160 bytes in 323 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:22.587132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:22.588863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4959645-e122-45b1-9ce9-e606be929360 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:22.661701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:22.663659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.664649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.665458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.680281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:22.683452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:22.704976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:22.707489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeb30f-bef7-4ae6-aab4-0c8d5a4ca6c2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:22.714406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:22.716085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:22.717874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:22.722977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 487/975] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:22.730513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:22.757428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:22.758542 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.758982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.759408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.768727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:22.770838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:22.788111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:22.789307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 13:48:22.790417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:22.792994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 489/976] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:48:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:22.799905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:22.888430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:22.935621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f58de-9f27-4d4f-b77e-3c6140e689e1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:22.937845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c0486-f839-45ff-a0e2-a9e97326cbd9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:22.981963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:22.982905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:22.983360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:22.984510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:22.985864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:22.987120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:22.990544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b83223-46f7-48a7-be21-f6196fb431ae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:23.009568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:23.010914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:23.011530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:23.012130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:23.014063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:23.014753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:23.017722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:23.040792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 488/977] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:23.044126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 490/978] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jul 15 13:48:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:23.054129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:23.129405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ebbc2-3997-4493-9cb0-cf08878a57cb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:23.130090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ebbc2-3997-4493-9cb0-cf08878a57cb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:23.131219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ebbc2-3997-4493-9cb0-cf08878a57cb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:23.133740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 491/979] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:23.139311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:23.139652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:23.140591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:23.164356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:23.166688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:23.186231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:23.200127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac882e-0c52-4030-8485-0f9a36e8fb9d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:23.252193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:23.253489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:23.253786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:23.254067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:23.255272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] RBAC: Adding query filter params (role.id=c03bc669e9e046598e003e38eb44dc89, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=9459ea72747e4ad4a839847d8232bb3c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:23.255590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:23.257499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf702a-156a-47db-bb26-0ae3cc6afc99 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:23.262669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:23.269301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 492/980] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:23.272606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 489/981] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:23 2022] GET /identity/v3/role_assignments?user.id=9459ea72747e4ad4a839847d8232bb3c&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=c03bc669e9e046598e003e38eb44dc89 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:23.293955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:23.441326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe4a80-23a5-451a-8dfa-e2ddd8fe6d08 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:23.483398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566c20a-42f6-4fcd-999d-b5f83cb570ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:23.484311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566c20a-42f6-4fcd-999d-b5f83cb570ba None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:23.484583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566c20a-42f6-4fcd-999d-b5f83cb570ba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:23.484863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566c20a-42f6-4fcd-999d-b5f83cb570ba None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:23.485621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566c20a-42f6-4fcd-999d-b5f83cb570ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:23.487036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566c20a-42f6-4fcd-999d-b5f83cb570ba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:23.504013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 493/982] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:48:23 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:24.634140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0e6f7-4446-411f-89ec-5c4eda902ba9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:24.634696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0e6f7-4446-411f-89ec-5c4eda902ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:24.635157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0e6f7-4446-411f-89ec-5c4eda902ba9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:24.637869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 490/983] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:24.643923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:24.644766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:24.644766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:24.663235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:24.664475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:24.667185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:24.760447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 494/984] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:24.783759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0e6f7-4446-411f-89ec-5c4eda902ba9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:24.906714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0e6f7-4446-411f-89ec-5c4eda902ba9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:24.947946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:24.948978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:24.949392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:24.949824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:24.950637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:24.952184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:24.965755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:24.967708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:24.970354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:24.972826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 491/985] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:24.976651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.094676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49572741-628f-4458-87bd-eff5bdb4fe2f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:25.135253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:25.135957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.136250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.136564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.137216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:25.137518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:25.138918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:25.140486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:25.152022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 495/986] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:25.159391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.237472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:25.284156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8022a0-09f7-4b56-ac3f-bb05fa815add None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:25.323875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:25.324581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.324872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.325128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.325849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:25.326152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:25.328108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060b9c7-2683-4006-83d9-6573221e2086 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:25.334221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 492/987] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:25.340317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.410064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a570887-1f19-44f0-bd2b-e584fea66e64 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.410317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a570887-1f19-44f0-bd2b-e584fea66e64 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.410513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a570887-1f19-44f0-bd2b-e584fea66e64 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.412096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 493/988] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:25.417384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.417673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.417962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.436143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:25.437680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:25.467721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5847e-44e7-4dec-8a2c-d14eb36aa5fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:25.513144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:25.514071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.514520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.515571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.516459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] RBAC: Adding query filter params (interface=admin, service_id=fed71697818b4a4fb551921f3a27bd6d, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:25.516805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:25.518375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 494/989] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:25.518556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:25.524885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 496/990] 10.222.0.16 () {56 vars in 1249 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/endpoints?service_id=fed71697818b4a4fb551921f3a27bd6d&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:25.530627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.541841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.624681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:25.624931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:25.679658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d89ee-33b0-475a-bd42-b12c5d7d2316 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:25.681349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ea8da-7907-415a-bf6d-c3060aeb3a60 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:25.728857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebb8ad-e911-4f5d-aa8e-fc9f19beba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:25.729553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebb8ad-e911-4f5d-aa8e-fc9f19beba9a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.729837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebb8ad-e911-4f5d-aa8e-fc9f19beba9a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.730116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebb8ad-e911-4f5d-aa8e-fc9f19beba9a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.730822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebb8ad-e911-4f5d-aa8e-fc9f19beba9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:25.731935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:25.732220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebb8ad-e911-4f5d-aa8e-fc9f19beba9a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:25.732787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.733071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.733348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.734018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:25.735398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:25.739501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 495/991] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/services => generated 1727 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 13:48:25.740864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318658a-e5b1-4464-9372-378f3e79e251 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.741154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318658a-e5b1-4464-9372-378f3e79e251 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.741431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318658a-e5b1-4464-9372-378f3e79e251 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.744458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 496/992] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:25.748315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:25.749922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:25.750732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 497/993] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/services/image => generated 86 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:25.752355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.752676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.752960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.756421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.774482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:25.775732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:25.886763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a3768-7947-44a5-88fb-752bf6d46d01 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:25.888789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 497/994] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:25.918055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.924218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:25.937323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:25.938373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:25.938373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:25.938869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:25.940108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:25.940108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:25.941996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:25.948011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 498/995] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:25.957669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:25.990424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:26.069217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e098ee8-d2f9-47b0-a50f-4c5fda2e550f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:26.087075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77d8dc-a4bf-422c-a980-c41a6583095f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:26.125457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:26.126275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:26.126553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:26.126828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:26.127486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:26.128920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:26.129808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:26.130491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:26.131000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:26.131281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:26.132054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:26.132320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:26.133775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:26.139442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 499/996] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:26.141560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:26.143611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:26.148517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:26.151039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 498/997] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:26.164583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:26.240807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:26.301589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5f048-f440-4f42-b18e-1e855f496d6c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:26.343133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:26.343835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:26.344120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:26.344419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:26.345187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:26.345497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:26.346930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:26.358465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 500/998] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:26.360421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:26.513044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989cdf5-7696-4410-8ad7-595efce3ed56 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:26.536574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:26.570823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:26.571968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:26.572529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:26.573041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:26.574249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:26.574721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:26.577460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:26.595263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 499/999] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:26.597704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:26.743256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449b0e1-802d-459b-8761-b7fdf58fa7dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:26.763802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:26.791841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:26.793052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:26.793582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:26.794138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:26.795776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] RBAC: Adding query filter params (interface=public, service_id=915c561380b5438896d49de7da7fc076, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:26.796923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:26.799606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de683b-6bc9-4b8f-b205-6125b5af345f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:26.816185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 501/1000] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:48:26 2022] GET /identity/v3/endpoints?service_id=915c561380b5438896d49de7da7fc076&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:26.818094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:26.957191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a8139-52e6-4f04-8790-a7a6f8287e85 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:26.997547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cae4b7-06e8-40df-b6e6-9d09e9def94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:26.998227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae4b7-06e8-40df-b6e6-9d09e9def94f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:26.998501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae4b7-06e8-40df-b6e6-9d09e9def94f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:26.998851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae4b7-06e8-40df-b6e6-9d09e9def94f None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:26.999593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cae4b7-06e8-40df-b6e6-9d09e9def94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:27.001082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cae4b7-06e8-40df-b6e6-9d09e9def94f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:27.009367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 500/1001] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:48:26 2022] GET /identity/v3/services => generated 1727 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 13:48:27.891741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7a1b8-c0eb-4e84-993f-d8f09a5de383 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:27.892551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7a1b8-c0eb-4e84-993f-d8f09a5de383 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:27.892551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7a1b8-c0eb-4e84-993f-d8f09a5de383 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:27.893721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 502/1002] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:27.900283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:27.900838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:27.901055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:27.920959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:27.922689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:27.987322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:28.032815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 501/1003] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:28.065300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7a1b8-c0eb-4e84-993f-d8f09a5de383 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:28.235073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7a1b8-c0eb-4e84-993f-d8f09a5de383 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:28.286183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:28.287099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.287386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.287659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.288492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:28.290382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-688e642e-cc2e-49dc-ac12-9af70a7bd413 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:28.313496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:28.314877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:28.321380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 503/1004] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jul 15 13:48:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:28.325761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:28.339049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6065b-7d28-4da2-9249-8686ae71f437 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.339402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6065b-7d28-4da2-9249-8686ae71f437 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.339676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6065b-7d28-4da2-9249-8686ae71f437 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.341195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 504/1005] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:28.346582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.346928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.347153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.365201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:28.366397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:28.418334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:28.458050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 505/1006] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:28.479829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b79993-b04d-4cbb-a08d-d7c9876f0a9e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:28.487389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:28.533007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:28.534077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.534555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.535014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.535889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:28.536366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:28.537938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:28.543846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 502/1007] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:28.552952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:28.606492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:28.643725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb70512-e7d2-464a-8524-bf21bb591250 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:28.688241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:28.688842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.689126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.689404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.690142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:28.691541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:28.716138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 506/1008] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:48:28 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:48:28.716465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e1c2c-1190-4b46-a7b9-f55723775ddf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:28.767090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:28.767777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.768060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.768338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.769121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:28.769428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:28.770871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060d766-9d9e-4426-aca6-278e20da1d19 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:28.777201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 503/1009] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jul 15 13:48:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:28.783884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:28.828571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:28.917298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c5329-2f75-4798-a359-fd38d189f105 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:28.959897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912991-d44b-40cb-a31d-2e9a9902baa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:28.960649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912991-d44b-40cb-a31d-2e9a9902baa4 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:28.960943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912991-d44b-40cb-a31d-2e9a9902baa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:28.961225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912991-d44b-40cb-a31d-2e9a9902baa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:28.961997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912991-d44b-40cb-a31d-2e9a9902baa4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:28.963470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912991-d44b-40cb-a31d-2e9a9902baa4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:28.982501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 507/1010] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:48:29.106402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ac6f9-1fe1-4ac9-85a1-24d72cea8bf1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:29.106959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ac6f9-1fe1-4ac9-85a1-24d72cea8bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:29.106959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ac6f9-1fe1-4ac9-85a1-24d72cea8bf1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:29.108702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 504/1011] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:29.115526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:29.116036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:29.116574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:29.146110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:29.148902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:29.272643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 508/1012] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:29 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:29.305046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ac6f9-1fe1-4ac9-85a1-24d72cea8bf1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:29.469843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:29.476108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ac6f9-1fe1-4ac9-85a1-24d72cea8bf1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:29.525742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:29.526467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:29.526741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:29.527011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:29.527687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:29.529143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:29.542897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:29.544031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:29.545389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 505/1013] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:29.551225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:29.588487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:29.704245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723d025-8e94-409d-bc83-fcd708305c34 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:29.744698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:29.745367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:29.745650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:29.745933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:29.746648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:29.746953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:29.748337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:29.754230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 509/1014] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:29.760452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:29.893055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd34e2-6455-45f7-8b16-262a4b35637f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:29.938653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:29.939422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:29.939750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:29.940027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:29.941056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:29.941508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:29.943101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b52642-a9b4-4490-9294-6d28bade130c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:29.956461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 506/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jul 15 13:48:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:29.961592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:30.110568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:30.147471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe923f-a80f-4c85-a315-189e64365f2d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:30.221805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87230509-1c5c-4ca1-829c-299d0ae529a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:30.223638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87230509-1c5c-4ca1-829c-299d0ae529a9 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:30.224443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87230509-1c5c-4ca1-829c-299d0ae529a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:30.225228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87230509-1c5c-4ca1-829c-299d0ae529a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:30.226937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87230509-1c5c-4ca1-829c-299d0ae529a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:30.229889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87230509-1c5c-4ca1-829c-299d0ae529a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:30.268617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 510/1016] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:30.980360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18876ed1-4b6e-4eb7-bb8a-d515ec435233 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:30.984620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18876ed1-4b6e-4eb7-bb8a-d515ec435233 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:30.984620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18876ed1-4b6e-4eb7-bb8a-d515ec435233 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:30.984620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 507/1017] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:30.990182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b005cf-2d31-4185-ba81-8de7fc71d817 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:30.991085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b005cf-2d31-4185-ba81-8de7fc71d817 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:30.991085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b005cf-2d31-4185-ba81-8de7fc71d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.009701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-62b005cf-2d31-4185-ba81-8de7fc71d817 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:31.011135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-62b005cf-2d31-4185-ba81-8de7fc71d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:31.097625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 511/1018] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:31.120576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18876ed1-4b6e-4eb7-bb8a-d515ec435233 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:31.157147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:31.219432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ada74f-7085-46ec-82cc-7064c4b79700 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.219432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ada74f-7085-46ec-82cc-7064c4b79700 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.219432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ada74f-7085-46ec-82cc-7064c4b79700 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.220803 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 512/1019] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:31.227934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.228565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.229039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.251094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:31.252601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:31.282321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18876ed1-4b6e-4eb7-bb8a-d515ec435233 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:31.333422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:31.334667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.334938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.335215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] PATH_INFO: `/v3/services/image` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.335898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:31.337339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:31.345109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:31.352281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:31.354505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:31.356198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 508/1020] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Jul 15 13:48:31 2022] GET /identity/v3/services/image => generated 86 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:31.360096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:31.364019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 513/1021] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:31.401374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:31.495372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:31.508476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3145d-dd0c-4455-9b22-1d7c7d9849bc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:31.551448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:31.552116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.552428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.552765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.553482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:31.553785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:31.555169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:31.564407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 509/1022] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:31.573821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:31.601371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1be6-664a-416a-b981-0c68d581d479 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:31.661414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:31.662475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.662932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.663382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.665210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:31.667915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:31.676532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 514/1023] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:31.680787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:31.751698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedadfd-c45c-41cf-8c84-d32d9b0266e2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:31.801961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:31.810383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:31.812610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.813152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.813639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.814563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:31.815069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:31.816744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:31.824630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 510/1024] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:31.830199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:31.865675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4858-c7bc-4662-ad17-4cb2422ac117 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:31.865936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:31.929576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:31.930460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:31.931203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:31.931614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:31.932809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:31.934655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:31.968941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 515/1025] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:48:31 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:32.022150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e034-d7b5-4ea4-8401-462c9b68f6b0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:32.091920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:32.093561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:32.094073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:32.094585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:32.096017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] RBAC: Adding query filter params (interface=public, service_id=6e6736323fdb4fcb966669407e7279bf, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:32.096617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:32.099596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42fe21-a5f6-45d8-a258-7f037f008ee0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:32.110412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 511/1026] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:48:31 2022] GET /identity/v3/endpoints?service_id=6e6736323fdb4fcb966669407e7279bf&interface=public®ion_id=RegionOne => generated 194 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:32.118835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:32.294110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79752ce-a493-4b2f-904f-c76e099dfca4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:32.315040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:32.338732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8859d48-c8fc-40e7-9038-dd75bff23b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:32.339414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8859d48-c8fc-40e7-9038-dd75bff23b07 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:32.339757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8859d48-c8fc-40e7-9038-dd75bff23b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:32.340021 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8859d48-c8fc-40e7-9038-dd75bff23b07 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:32.340734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8859d48-c8fc-40e7-9038-dd75bff23b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:32.342213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8859d48-c8fc-40e7-9038-dd75bff23b07 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:32.351251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 516/1027] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:48:32 2022] GET /identity/v3/services => generated 1957 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 13:48:33.140478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ef6d-5945-42d7-a826-f890ef0b4370 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:33.141329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ef6d-5945-42d7-a826-f890ef0b4370 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:33.141485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ef6d-5945-42d7-a826-f890ef0b4370 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:33.146940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 512/1028] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:33.153600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:33.154343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:33.154662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:33.177858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:33.179059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:33.260263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 517/1029] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:33.284555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ef6d-5945-42d7-a826-f890ef0b4370 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:33.445213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:33.472235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ef6d-5945-42d7-a826-f890ef0b4370 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:33.512712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:33.513559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:33.513847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:33.514128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:33.527893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:33.536851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:33.538735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:33.541318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 513/1030] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jul 15 13:48:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:33.548765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:33.743728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31368b-bf04-4e14-88a2-097b75b50d88 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:33.770699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:33.816545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:33.817728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:33.818242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:33.818751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:33.820053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:33.820652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:33.823346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:33.839608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 518/1031] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:48:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:33.848995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.015649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372307aa-db5a-42c0-b8b3-94cc60ce983e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:34.057380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:34.058185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.058470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.058746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.070341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:34.071818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:34.073890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e68e8252-d337-4cd9-b684-c9e7e74d789e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:34.098458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:34.099754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:34.110511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:34.110511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:34.110511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:34.110511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:34.110511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 514/1032] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:34.111111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.189716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40911-d39d-4f3e-b525-3ad9c566ea22 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.190766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40911-d39d-4f3e-b525-3ad9c566ea22 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.190766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf40911-d39d-4f3e-b525-3ad9c566ea22 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.192582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 515/1033] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:34.198261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.198545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.198881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.217702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:34.218932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:34.286863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dd22d-11a8-43e1-a5e6-42de4a35d8dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:34.352997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 516/1034] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:34.357177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:34.358999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.359507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.360019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.361522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:34.362078 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:34.364842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:34.383612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.399345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 519/1035] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:34.401451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.464779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:34.526242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3ace-fd0b-4768-b41a-9b17cbda975e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:34.560053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:34.565159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48ed6e-c3a2-414e-bb07-ffe4a637bf62 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:34.573679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb37b5e-52a4-44ee-9dba-62a3e9e4ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:34.574529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37b5e-52a4-44ee-9dba-62a3e9e4ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.574807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37b5e-52a4-44ee-9dba-62a3e9e4ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.575089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37b5e-52a4-44ee-9dba-62a3e9e4ddd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.576488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb37b5e-52a4-44ee-9dba-62a3e9e4ddd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:34.578013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb37b5e-52a4-44ee-9dba-62a3e9e4ddd6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:34.579773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 517/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:34.581183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651ba2-e6d2-465d-a1c6-7819ea1b8f5b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.581468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651ba2-e6d2-465d-a1c6-7819ea1b8f5b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.581747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651ba2-e6d2-465d-a1c6-7819ea1b8f5b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.583155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 518/1037] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:34.587717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9651ba2-e6d2-465d-a1c6-7819ea1b8f5b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.619464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:34.621122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.621652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.622138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.636659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:34.639922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-31d6a9bb-c0da-4dfe-9463-6a444bba3c74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:34.670407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:34.673020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:48:34.675196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:34.678891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 520/1038] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:48:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:34.681787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.682557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.683277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.718388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:34.720884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:34.738876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9651ba2-e6d2-465d-a1c6-7819ea1b8f5b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:34.778063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:34.778735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:34.779013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:34.779286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:34.780013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:34.782011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:34.808792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 519/1039] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:48:34 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:34.809914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.828483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 521/1040] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:34.853715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:34.897246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:34.910207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:35.006583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7577f7-4fc6-41d7-ae29-581af897467c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.012374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9cd7-1ffb-4c9a-9a99-ddf374c4357a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.058618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.059384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.059656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.059979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.060927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:35.061237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.062735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.069396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.070210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.070489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.070753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.071426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.072876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:35.086714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:35.088695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:35.090372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 522/1041] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Jul 15 13:48:34 2022] GET /identity/v3/services/image => generated 86 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:35.091937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 520/1042] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:48:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:35.095879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:35.099181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:35.231071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2efed-fa25-486f-b432-f9cc3a5e5aa0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.234205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:35.242635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:35.254449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e3157-209c-4c50-a0db-b3b7f3607a3e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.282771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.283894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.284322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.284677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.307487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.308664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.309176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.309607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.310719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:35.311280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.313742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.324222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 523/1043] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:35.334225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:35.378509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.379909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.486591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05cdfa-b229-4557-9f63-10564c3599ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.529788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.531260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.531631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.531989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.532503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 521/1044] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:35.532945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:35.533459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.535261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.541047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:35.544846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 524/1045] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:35.551495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:35.552092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:35.571131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:35.681317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d247-f000-4659-bbff-c9e63173916b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.716580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d294b-afe0-4b04-a483-8f9780a04cfe None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:35.722127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.723536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.723820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.724096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.725060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.727549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.764419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 525/1046] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:48:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:48:35.794285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:35.795765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:35.797111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:35.797574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:35.811247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:35.812795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:35.834308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:35.835611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:35.836715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:35.836715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:35.836715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:35.836715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:35.843064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 522/1047] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:35.850657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:35.950509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:36.001618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd557102-8f2c-4ba2-b3c4-7dd1db7c5f60 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:36.039830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:36.040530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:36.040813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:36.041084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:36.041830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:36.042134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:36.043535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:36.057272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 526/1048] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:36.064050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:36.132231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:36.210882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f0a80-69ed-4362-8aa6-5d1344c4a679 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:36.276988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:36.278453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:36.278956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:36.279464 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:36.298716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:36.301592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:36.330285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:36.331293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:36.332205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:36.333422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 523/1049] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:36.341763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:36.485469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c86189-9a86-4c30-9c8f-02637961092c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:36.520825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:36.528652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:36.529324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:36.529604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:36.529882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:36.530805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:36.531104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:36.532617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406115e-c5b3-4dce-be5b-81ddb03c21ae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:36.560441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 527/1050] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:36.564284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:36.751675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba824c9-3d6b-452c-ae32-5a49780522d0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:36.777297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:36.815932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:36.817134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:36.817652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:36.818175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:36.819865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] RBAC: Adding query filter params (role.id=c03bc669e9e046598e003e38eb44dc89, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=e424ea6d03414857bc408e49b8df5429) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:36.820426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:36.823096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b93ddd-7c43-4e0b-92e9-12a84fe9d955 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:36.840427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 524/1051] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:36 2022] GET /identity/v3/role_assignments?user.id=e424ea6d03414857bc408e49b8df5429&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=c03bc669e9e046598e003e38eb44dc89 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:37.138651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b8aa-8534-46c6-8c2c-5114e25712d0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:37.138974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b8aa-8534-46c6-8c2c-5114e25712d0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:37.139197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b8aa-8534-46c6-8c2c-5114e25712d0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:37.141851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 528/1052] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:37.147589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:37.147857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:37.148153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:37.174699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:37.176148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:37.304560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 525/1053] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:37.343686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b8aa-8534-46c6-8c2c-5114e25712d0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:37.531584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b8aa-8534-46c6-8c2c-5114e25712d0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:37.594906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:37.596291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:37.596837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:37.597354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:37.617743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:37.633298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:37.634295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:37.635132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:37.636478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 529/1054] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:37.645965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:37.819338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efbcb5-4179-4888-8d66-0a58ca39c5b3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:37.837720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:37.880529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:37.881233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:37.881533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:37.881864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:37.882799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:37.883308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:37.885201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d19430-5f89-4663-9989-7235e64fdc7d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:37.897320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 526/1055] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:37.904459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:38.002588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:38.093090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e510db-36ea-4530-9248-c5b2289f0878 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:38.095984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71652fea-aff6-409f-8e9f-1382ef2872b3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.096540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71652fea-aff6-409f-8e9f-1382ef2872b3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.097057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71652fea-aff6-409f-8e9f-1382ef2872b3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.099829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 527/1056] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:38.105624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.106115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.106409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.130207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:38.132041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:38.160436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:38.161464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.161842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.162195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] PATH_INFO: `/v3/users/98f95946999947fabf6d84534dea186c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.180971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] RBAC: Authorizing `identity:get_user(user_id=98f95946999947fabf6d84534dea186c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:38.182431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:38.184435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 530/1057] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:37 2022] GET /identity/v3/users/98f95946999947fabf6d84534dea186c => generated 285 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:38.190965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:38.261092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 528/1058] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:38.277723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:38.321738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a59e1-b339-4e4f-9c9c-0a231e8700d4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:38.335435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:38.363936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:38.364954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.364954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.365203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.401483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:38.406244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8579f-cd3b-4968-ab53-86e6edb22c6c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:38.445226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:38.447013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:38.447171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c590f48-5101-492c-994b-083e25362fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:38.447755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c590f48-5101-492c-994b-083e25362fef None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.448046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c590f48-5101-492c-994b-083e25362fef None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.448323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c590f48-5101-492c-994b-083e25362fef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.538484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c590f48-5101-492c-994b-083e25362fef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:38.540334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c590f48-5101-492c-994b-083e25362fef None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:38.581539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 531/1059] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:38.587410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:38.600712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:38.680941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:38.692233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 529/1060] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:38.693697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c590f48-5101-492c-994b-083e25362fef None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:38.708382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e281-1ce6-4e38-9b95-326726aa4eaf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:38.766434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:38.767808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.768221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.768667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.770681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7394674f44c199d99f3f6e8f17bfc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:38.773056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:38.781397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 532/1061] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:48:38 2022] GET /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:38.785858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:38.879668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c590f48-5101-492c-994b-083e25362fef None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:38.907165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:38.939592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:38.940326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.940659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.940936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.941972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72f426-52e3-4901-9194-b040d01889db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:38.953368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:38.954882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e3e98faa-20b6-487b-9c8f-2b9d7c74fdac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:38.974851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:38.976750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 530/1062] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32b1a-23e4-4d0b-b4cc-132eb103b996 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32b1a-23e4-4d0b-b4cc-132eb103b996 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.978749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32b1a-23e4-4d0b-b4cc-132eb103b996 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:38.979681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 531/1063] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:38.985617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:38.985909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:38.986192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:39.003556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:39.004212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:39.004422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:39.004696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:39.005020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:39.005496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:39.005992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc, user.id=98f95946999947fabf6d84534dea186c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:39.006346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:39.007979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:39.015351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 533/1064] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:38 2022] GET /identity/v3/role_assignments?user.id=98f95946999947fabf6d84534dea186c&scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:39.016436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:39.074491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:39.104507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 532/1065] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:39.127119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:39.164378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327bce7-e8b3-488b-8c70-c4b6f33600b8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:39.221020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:39.221624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:39.221926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:39.222225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:39.222907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:39.223475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:39.224909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:39.237825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 534/1066] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:39.279116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abd931-6fad-40e3-8dd2-3754e747aed8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:39.324906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:39.326002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:39.326435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:39.326978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:39.341835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:39.351002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:39.352043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:39.353528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:39.356882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 533/1067] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jul 15 13:48:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:39.358725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:39.362729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:39.512233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685f538-71cd-4e11-bab8-5c73a03f955f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:39.578438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:39.579577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:39.580085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:39.580622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:39.582034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:39.582586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:39.585030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:39.600427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 535/1068] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:48:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:39.606248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:39.652936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:39.718948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e953b5-0943-4619-bb9b-1ea204b6bd46 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:39.761145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:39.761962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:39.762317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:39.762501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:39.776220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:39.778007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:39.799434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:39.801260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:39.802964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:39.802964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:39.802964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:39.802964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:39.802964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 534/1069] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:39.808841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:39.885159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:39.946051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22463fb5-a0f2-4aef-917b-bfb4b2e35990 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:40.008620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:40.009770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:40.010265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:40.010755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:40.012105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:40.012734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:40.015806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:40.042992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 536/1070] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:40.044359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:40.157526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:40.167476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50ff02-3a25-460a-87d0-ff23654e2e07 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:40.206457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:40.207383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:40.207757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:40.208125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:40.217434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:40.219108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:40.235419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:40.236926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:40.238642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 535/1071] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:48:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:40.243074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:40.372737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f4a1e-3931-46b8-8798-95544df01141 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:40.403734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:40.417634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:40.418406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:40.418752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:40.419095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:40.420154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:40.420569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:40.422650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:40.443630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 537/1072] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:48:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:40.450288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:40.577092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8b3b-a514-4c2b-a704-c839c13bc8a0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:40.635423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:40.636161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:40.636649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:40.637103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:40.655101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:40.657923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:40.691225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:40.692266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:40.693232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:40.693232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:40.693232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:40.693232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:40.698418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 536/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:40.699803 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:40.876094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e2d52-8e80-444b-930d-7d4762a0cf9e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:40.936615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:40.937451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:40.937762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:40.938258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:40.939666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:40.940181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:40.942818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:40.956819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 538/1074] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:40.962636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:40.973783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:41.083117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c941d4-8695-4bee-b12e-e768cdaccb39 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:41.121765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:41.122534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.122810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.123082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.130743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:41.132184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-38c0a6e8-52d5-4f33-9727-eb1a8eaca6a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:41.147639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:41.149617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:41.153013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 537/1075] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:41.156024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:41.235781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:41.321703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b853a-cf6c-4930-bfaa-1bd4aa70e2a0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.322204 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b853a-cf6c-4930-bfaa-1bd4aa70e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.322719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b853a-cf6c-4930-bfaa-1bd4aa70e2a0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.325510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 538/1076] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:41.332353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.332873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.333339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.333777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97d0a2-8063-4c9f-b666-1f636487313e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:41.355129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:41.356361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:41.373230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:41.374445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.374719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.374987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.375781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:41.376088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:41.377664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:41.391037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 539/1077] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:41.398266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:41.429960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:41.438978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 539/1078] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:41.461676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:41.536815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c443e9-cb97-4f98-be1c-eb286f35d095 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:41.538425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:41.577390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:41.578291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.578573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.578850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/e424ea6d03414857bc408e49b8df5429/roles/c03bc669e9e046598e003e38eb44dc89` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.581290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=e424ea6d03414857bc408e49b8df5429, role_id=c03bc669e9e046598e003e38eb44dc89)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:41.582754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:41.601457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 540/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:48:41 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/e424ea6d03414857bc408e49b8df5429/roles/c03bc669e9e046598e003e38eb44dc89 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 13:48:41.627973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a1d0f-a1b4-42a6-a592-7f62f4f9683a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:41.693945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:41.694770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.695054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.695330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.707639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:41.715603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:41.716764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:41.718589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 540/1080] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:48:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:41.724607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:41.849759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3716c-3f0d-4dc3-b519-2a89811d6ee4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:41.904625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:41.906347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:41.906706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:41.907201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:41.908670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:41.909234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:41.911638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:41.932528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 541/1081] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:48:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:41.934810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:42.046201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:42.130760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a0f41-b8c1-4f92-8d5b-361601d1c772 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:42.201384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:42.203398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:42.203870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:42.204379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] PATH_INFO: `/v3/users/98f95946999947fabf6d84534dea186c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:42.206520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] RBAC: Authorizing `identity:get_user(user_id=98f95946999947fabf6d84534dea186c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:42.209145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:42.217026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 541/1082] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:41 2022] GET /identity/v3/users/98f95946999947fabf6d84534dea186c => generated 285 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:42.218544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:42.250949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:42.351815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6856-c1b1-4a0a-a8f9-36e20960374d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:42.415500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097443e-d267-4707-82c1-9ead6ebf748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:42.417006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097443e-d267-4707-82c1-9ead6ebf748c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:42.417517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097443e-d267-4707-82c1-9ead6ebf748c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:42.418016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097443e-d267-4707-82c1-9ead6ebf748c None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:42.420180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097443e-d267-4707-82c1-9ead6ebf748c None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7394674f44c199d99f3f6e8f17bfc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:42.422773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097443e-d267-4707-82c1-9ead6ebf748c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:42.431054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 542/1083] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:48:42 2022] GET /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc => generated 311 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:42.432991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:42.580839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856312dd-0078-4b5f-b04b-ca2736e9841c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:42.607430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:42.621626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ed9c7-698c-4e45-b8b2-b69e6402a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:42.622658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed9c7-698c-4e45-b8b2-b69e6402a9ac None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:42.623025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed9c7-698c-4e45-b8b2-b69e6402a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:42.623413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ed9c7-698c-4e45-b8b2-b69e6402a9ac None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc/users/98f95946999947fabf6d84534dea186c/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:42.626359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ed9c7-698c-4e45-b8b2-b69e6402a9ac None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d7394674f44c199d99f3f6e8f17bfc, user_id=98f95946999947fabf6d84534dea186c, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:42.627917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ed9c7-698c-4e45-b8b2-b69e6402a9ac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:42.643243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 542/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:48:42 2022] PUT /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc/users/98f95946999947fabf6d84534dea186c/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:48:43.792424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb060-75c6-4282-aacc-e33e634a09c7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:43.793085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb060-75c6-4282-aacc-e33e634a09c7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:43.793558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eb060-75c6-4282-aacc-e33e634a09c7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:43.796110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 543/1085] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:43.803311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:43.803902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:43.804272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:43.833781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:43.836048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:43.952531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 543/1086] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:43.985146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb060-75c6-4282-aacc-e33e634a09c7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:44.095536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:44.111614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eb060-75c6-4282-aacc-e33e634a09c7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:44.153020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:44.153834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.154106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.154374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.173368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:44.189975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:44.191115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 13:48:44.191928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:44.193132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 544/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jul 15 13:48:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:44.202650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:44.321567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ffb98-600d-40e6-ac5c-c57f64e74fd5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:44.338921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:44.365882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:44.366612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.366977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.367341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.368201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:44.368629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:44.370124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:44.379651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 544/1088] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:48:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:44.385463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:44.496632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93bf88-a80a-4c4c-a550-cfded3a44905 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:44.531833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:44.532654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.532937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.533213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.544713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:44.546206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:44.564613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:44.566366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:44.568147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:44.568147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:44.568147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:44.568147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:44.568147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 545/1089] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:44.572898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:44.676467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51679b3-9dd5-4f9f-9f8c-60a870ac415c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:44.713960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:44.714627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.714910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.715186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.715917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:44.716224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:44.717720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb273c1-9d66-4642-b3de-316f2ee63627 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:44.730938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 545/1090] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:44.736877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:44.741096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:44.813914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aae04-64d3-48d0-a016-c112a7173636 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.814479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aae04-64d3-48d0-a016-c112a7173636 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.814479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aae04-64d3-48d0-a016-c112a7173636 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.815864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 546/1091] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:44.821315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.821606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.821886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.838840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:44.840038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:44.844292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c18f1-cacb-461f-8634-f47cb4ce65aa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:44.871661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:44.882798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:44.883601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:44.883883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:44.884157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:44.892339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:44.893837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:48:44.908891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:44.915819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:44.916845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 546/1092] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jul 15 13:48:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:44.917522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 547/1093] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:44.917604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:44.952861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.011036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.053018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f95c5d-3e8e-4966-bf02-89f1325db838 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.063318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce886fda-e11c-4757-a188-1849a109c6b9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.089241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.090086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.090542 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.090945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.092136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:45.092720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.094426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.101401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.102232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.102526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.102796 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.118263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:48:45.118958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 547/1094] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jul 15 13:48:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:48:45.127268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:45.128503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:48:45.129478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:48:45.130320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.130399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 548/1095] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:48:45.134956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.169176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.241566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c075cb-074b-4d7c-94cd-9898526247e3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.265691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.278652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.279299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.279576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.280467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.297116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149e4b3-307a-4612-a0d5-565a7f6502fb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.342071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.343297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.343580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.343859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.344643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:45.344953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.346359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.356444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 549/1096] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:45.364311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.365817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.366368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.487185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 548/1097] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:45.490492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.493253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.507867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc56654-84fa-4a2c-90da-d3dee6a783c6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.520301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.550736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.551901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.552269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.552660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] PATH_INFO: `/v3/users/98f95946999947fabf6d84534dea186c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.554062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] RBAC: Authorizing `identity:get_user(user_id=98f95946999947fabf6d84534dea186c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.555998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.558243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 550/1098] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/users/98f95946999947fabf6d84534dea186c => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:45.566964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.608008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0206e-10d2-4a04-b389-f39cb2b76244 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.658219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.659029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.659308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.659585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.671405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.672938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:48:45.691856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:45.693507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:45.695135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:48:45.695135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:48:45.695135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:48:45.695135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:45.695135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 549/1099] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:45.699223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.700322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aa7b1-f222-4574-939c-63c411bef785 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.744964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.745609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.746001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.746399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.782178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.802933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:45.838889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d333e8-e068-49f7-974b-cb7254f30beb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:45.840721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.843141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.889476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:45.890640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:45.891149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:45.891660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:45.893108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:45.893678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:45.896310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:45.918431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 550/1100] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:45.925586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:45.976150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 551/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:45.982663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:46.052402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:46.054968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf020-64c5-4b49-bc16-3981f1a5f0c9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:46.098644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:46.099457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:46.099737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:46.100011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:46.102296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:46.109503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:46.109699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f85c1-5de4-4d22-b724-fbd21417611c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:46.111495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:48:46.125833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:48:46.126982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:48:46.127851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 551/1102] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:48:46.137691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:46.149845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:46.150721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:46.151000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:46.151300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:46.152535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7394674f44c199d99f3f6e8f17bfc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:46.153983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:46.156305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 552/1103] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:48:45 2022] GET /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:46.162738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:46.275257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c77f-021d-4a9a-9cad-beccad38895e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:46.284061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29762c7a-c0b0-408e-a393-80741a394529 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:46.312768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:46.315640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:46.316311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:46.316614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:46.316889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:46.317697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:46.317997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:46.319469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:46.328477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:46.329142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:46.329431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:46.329704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:46.330540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc, user.id=98f95946999947fabf6d84534dea186c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:46.330905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:46.332352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eec9e7-e531-4d18-9408-f6b7a0bad104 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:46.335114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 552/1104] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:46.341278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:46.350209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 553/1105] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:46 2022] GET /identity/v3/role_assignments?user.id=98f95946999947fabf6d84534dea186c&scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:46.412363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:46.470900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0765551-5ffb-4a9e-9023-98df5bf345f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:46.517422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:46.518080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:46.518358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:46.518639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:46.519463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] RBAC: Adding query filter params (role.id=c03bc669e9e046598e003e38eb44dc89, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=e424ea6d03414857bc408e49b8df5429) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:46.519769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:46.521240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e036833-6787-45c5-9dcb-dfe7e6aae0a7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:46.527992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 553/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:46 2022] GET /identity/v3/role_assignments?user.id=e424ea6d03414857bc408e49b8df5429&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=c03bc669e9e046598e003e38eb44dc89 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:48.488331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02211c3-99d8-4386-8ab1-b623289de8d0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:48.488897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02211c3-99d8-4386-8ab1-b623289de8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:48.488897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02211c3-99d8-4386-8ab1-b623289de8d0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:48.490242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 554/1107] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:48:48.496289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:48.496844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:48.496976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:48.514205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:48.515420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:48.588496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 554/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:48.617939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02211c3-99d8-4386-8ab1-b623289de8d0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:48.729526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02211c3-99d8-4386-8ab1-b623289de8d0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:48.766501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8039-837e-4e13-838e-7778c62da9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:48.767214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8039-837e-4e13-838e-7778c62da9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:48.767497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8039-837e-4e13-838e-7778c62da9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:48.767774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8039-837e-4e13-838e-7778c62da9f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:48.769011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8039-837e-4e13-838e-7778c62da9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:48.770467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8039-837e-4e13-838e-7778c62da9f4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:48.772119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 555/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:48.779135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:48.858068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:48.915117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bb4c9-6f0d-4b3c-bbf7-2c2d24006a22 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:48.955815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef5ee-fdc7-47b9-b2b0-732612a9eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:48.956584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef5ee-fdc7-47b9-b2b0-732612a9eca8 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:48.956920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef5ee-fdc7-47b9-b2b0-732612a9eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:48.957248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef5ee-fdc7-47b9-b2b0-732612a9eca8 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:48.958033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ef5ee-fdc7-47b9-b2b0-732612a9eca8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:48.959538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ef5ee-fdc7-47b9-b2b0-732612a9eca8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:48.985957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 555/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:48:48 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:48:51.078432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534ce54-8a9c-432f-a1c1-95b3ffbac2f5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:51.078997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534ce54-8a9c-432f-a1c1-95b3ffbac2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:51.078997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534ce54-8a9c-432f-a1c1-95b3ffbac2f5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:51.080512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 556/1111] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:51.086501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:51.087016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:51.087105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:51.105573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:51.106902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:51.180635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 556/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:48:51.205280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534ce54-8a9c-432f-a1c1-95b3ffbac2f5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:51.292196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:51.362738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534ce54-8a9c-432f-a1c1-95b3ffbac2f5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:51.405477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:51.406908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:51.407334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:51.407759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] PATH_INFO: `/v3/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:51.410268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] RBAC: Authorizing `identity:get_role(role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:51.412793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:51.415487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 557/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:51 2022] GET /identity/v3/roles/c41b65addaec48bfb92f135a2a724132 => generated 223 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:51.421835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:51.461050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:51.539984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0803129-c9b9-489f-bbdf-d0ab9884d8d2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:51.578515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:51.579427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:51.579713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:51.579997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] PATH_INFO: `/v3/users/0b34a7dc60e44ae595fae616e0bc52ba` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:51.596620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] RBAC: Authorizing `identity:get_user(user_id=0b34a7dc60e44ae595fae616e0bc52ba)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:51.598380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:51.600204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 557/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:51 2022] GET /identity/v3/users/0b34a7dc60e44ae595fae616e0bc52ba => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:48:51.606582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:51.716357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c9fd5-0460-474f-ae09-8b8142706482 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:51.742931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:51.755650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:51.756319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:51.756637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:51.756915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:51.835175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:51.836815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:51.963275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 558/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:48:51.969928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:52.077825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:52.087403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd0514-f4b7-4a61-bf37-6404516ca5cd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:52.122503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e092fbe-1f77-4e5b-b3c6-93da74f4477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:52.123373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e092fbe-1f77-4e5b-b3c6-93da74f4477a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:52.124196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e092fbe-1f77-4e5b-b3c6-93da74f4477a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:52.124499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e092fbe-1f77-4e5b-b3c6-93da74f4477a None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:52.125741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e092fbe-1f77-4e5b-b3c6-93da74f4477a None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7394674f44c199d99f3f6e8f17bfc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:52.127182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e092fbe-1f77-4e5b-b3c6-93da74f4477a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:52.128990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 558/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:48:51 2022] GET /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:52.135316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:52.163437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:52.245878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91271ab-be87-4edc-9e33-4c828cd210fc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:52.284517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:52.285189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:52.285475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:52.285754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:52.286599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] RBAC: Adding query filter params (role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc, user.id=0b34a7dc60e44ae595fae616e0bc52ba) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:52.286914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:52.288371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd2e7b-182e-49ac-9ade-445cb8ae2140 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:52.294491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 559/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:52 2022] GET /identity/v3/role_assignments?user.id=0b34a7dc60e44ae595fae616e0bc52ba&scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc&role.id=c41b65addaec48bfb92f135a2a724132 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:54.391926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03029982-d43d-45ae-aab4-8207016a4d65 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:54.392578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03029982-d43d-45ae-aab4-8207016a4d65 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:54.392982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03029982-d43d-45ae-aab4-8207016a4d65 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:54.395594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 559/1118] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:54.403902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:54.404528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:54.404528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:54.421582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:54.422705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:54.495773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 560/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:54.524077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03029982-d43d-45ae-aab4-8207016a4d65 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:54.641926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03029982-d43d-45ae-aab4-8207016a4d65 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:54.684462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:54.685264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:54.685545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:54.685819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] PATH_INFO: `/v3/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:54.688098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] RBAC: Authorizing `identity:get_role(role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:54.689546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:54.691037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 560/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:54 2022] GET /identity/v3/roles/c41b65addaec48bfb92f135a2a724132 => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:54.697380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:54.718357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:54.808905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81b170-9eed-405a-9c21-907bd8b9f2d9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:54.844844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:54.845722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:54.846005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:54.846279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] PATH_INFO: `/v3/users/0b34a7dc60e44ae595fae616e0bc52ba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:54.847528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] RBAC: Authorizing `identity:get_user(user_id=0b34a7dc60e44ae595fae616e0bc52ba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:54.849281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:54.850963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 561/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:54 2022] GET /identity/v3/users/0b34a7dc60e44ae595fae616e0bc52ba => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:54.857185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:54.888062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:54.969731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105d3da-ac68-4915-b54a-40020e7d1961 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:55.012173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ce5a-aaa5-41d2-96d2-862e5e1f4dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:55.013126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ce5a-aaa5-41d2-96d2-862e5e1f4dab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:55.013413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ce5a-aaa5-41d2-96d2-862e5e1f4dab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:55.013753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ce5a-aaa5-41d2-96d2-862e5e1f4dab None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:55.015287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10ce5a-aaa5-41d2-96d2-862e5e1f4dab None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7394674f44c199d99f3f6e8f17bfc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:55.016806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10ce5a-aaa5-41d2-96d2-862e5e1f4dab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:55.018670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 561/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:48:54 2022] GET /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:55.025049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:55.125474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:55.142521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5bf59-ad1d-4d5b-80e8-7919104ffed4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:55.180365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489610c-66cd-4cc4-b105-d8ae5e8055e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:55.181289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489610c-66cd-4cc4-b105-d8ae5e8055e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:55.181574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489610c-66cd-4cc4-b105-d8ae5e8055e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:55.181848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489610c-66cd-4cc4-b105-d8ae5e8055e6 None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc/users/0b34a7dc60e44ae595fae616e0bc52ba/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:55.184104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489610c-66cd-4cc4-b105-d8ae5e8055e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d7394674f44c199d99f3f6e8f17bfc, user_id=0b34a7dc60e44ae595fae616e0bc52ba, role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:55.185601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489610c-66cd-4cc4-b105-d8ae5e8055e6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:55.200037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 562/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:48:55 2022] PUT /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc/users/0b34a7dc60e44ae595fae616e0bc52ba/roles/c41b65addaec48bfb92f135a2a724132 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:48:57.259229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae7a5-e124-4d7f-ba3b-c5497b4e5115 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:57.259825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae7a5-e124-4d7f-ba3b-c5497b4e5115 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:57.259825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae7a5-e124-4d7f-ba3b-c5497b4e5115 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:57.261230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 562/1124] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:48:57.270410 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:57.270930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:57.271386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:57.297297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:48:57.299353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:48:57.394378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 563/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:48:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:48:57.425423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae7a5-e124-4d7f-ba3b-c5497b4e5115 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:57.530375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:57.537685 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae7a5-e124-4d7f-ba3b-c5497b4e5115 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:57.573500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:57.574342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:57.574621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:57.574891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] PATH_INFO: `/v3/roles/c41b65addaec48bfb92f135a2a724132` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:57.576427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] RBAC: Authorizing `identity:get_role(role_id=c41b65addaec48bfb92f135a2a724132)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:57.577811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:57.579220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 563/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:57 2022] GET /identity/v3/roles/c41b65addaec48bfb92f135a2a724132 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:57.587094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:57.669180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:57.696031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22009006-675f-42a9-8721-5ecd2c7a711e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:57.733517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:57.734404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:57.734684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:57.734955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] PATH_INFO: `/v3/users/0b34a7dc60e44ae595fae616e0bc52ba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:57.736220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] RBAC: Authorizing `identity:get_user(user_id=0b34a7dc60e44ae595fae616e0bc52ba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:57.737700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:57.739384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 564/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:48:57 2022] GET /identity/v3/users/0b34a7dc60e44ae595fae616e0bc52ba => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:57.749195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:57.883182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feabf59-24b5-492d-9962-cd5ca2a4348f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:57.924632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:57.925306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:57.925588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:57.925866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:57.963341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:58.015414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:58.016946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:58.130888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 564/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:48:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:48:58.137066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:58.246598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c17cc-8363-4786-8c47-4d07945fface None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:58.285221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9662ad-6a9b-4fb8-a3f3-dcffba35a9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:58.286153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9662ad-6a9b-4fb8-a3f3-dcffba35a9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:58.286486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9662ad-6a9b-4fb8-a3f3-dcffba35a9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:58.286810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9662ad-6a9b-4fb8-a3f3-dcffba35a9f6 None admin] PATH_INFO: `/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:58.288103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9662ad-6a9b-4fb8-a3f3-dcffba35a9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7394674f44c199d99f3f6e8f17bfc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:58.289657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9662ad-6a9b-4fb8-a3f3-dcffba35a9f6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:58.291478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 565/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:48:58 2022] GET /identity/v3/projects/e9d7394674f44c199d99f3f6e8f17bfc => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:48:58.298555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:48:58.364870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:48:58.411269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d20d16-d93a-4e63-be5f-1536253e826b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:48:58.448055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:48:58.448762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:48:58.449045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:48:58.449327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:48:58.450198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] RBAC: Adding query filter params (role.id=c41b65addaec48bfb92f135a2a724132, scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc, user.id=0b34a7dc60e44ae595fae616e0bc52ba) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:48:58.450516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:48:58.452569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cbc6d-3088-481f-8b53-83d1f5396313 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:48:58.459606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 565/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:48:58 2022] GET /identity/v3/role_assignments?user.id=0b34a7dc60e44ae595fae616e0bc52ba&scope.project.id=e9d7394674f44c199d99f3f6e8f17bfc&role.id=c41b65addaec48bfb92f135a2a724132 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:00.555513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bc182-d606-44ed-80df-34907f9259d3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:00.556078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bc182-d606-44ed-80df-34907f9259d3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:00.556078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bc182-d606-44ed-80df-34907f9259d3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:00.557513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 566/1131] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:00.563739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:00.564260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:00.564425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:00.581604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:00.582822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:00.656088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 566/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:49:00.679828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bc182-d606-44ed-80df-34907f9259d3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:00.695750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:00.798397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bc182-d606-44ed-80df-34907f9259d3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:00.845434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3887a0-2ef9-4633-9edc-bf1cf7a25ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:00.846129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3887a0-2ef9-4633-9edc-bf1cf7a25ddd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:00.846417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3887a0-2ef9-4633-9edc-bf1cf7a25ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:00.846697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3887a0-2ef9-4633-9edc-bf1cf7a25ddd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:00.847938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3887a0-2ef9-4633-9edc-bf1cf7a25ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:00.849683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3887a0-2ef9-4633-9edc-bf1cf7a25ddd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:00.851354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 567/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:00.858213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:00.927288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:00.971830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4aa5a-a9bd-4a92-9b18-fcddf930216b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:01.008626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48838-05af-4c6a-b149-4b8791f83854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:01.009449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48838-05af-4c6a-b149-4b8791f83854 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:01.009733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48838-05af-4c6a-b149-4b8791f83854 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:01.010014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48838-05af-4c6a-b149-4b8791f83854 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:01.010745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d48838-05af-4c6a-b149-4b8791f83854 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:01.012194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d48838-05af-4c6a-b149-4b8791f83854 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:01.039691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 567/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:49:00 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:49:03.188407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca902b8-97a8-4a9e-9d75-a22ac6ca8804 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:03.188995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca902b8-97a8-4a9e-9d75-a22ac6ca8804 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:03.189388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca902b8-97a8-4a9e-9d75-a22ac6ca8804 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:03.191914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 568/1135] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:03.197781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:03.198330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:03.198471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:03.216274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:03.217560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:03.291385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 568/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:49:03.316584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca902b8-97a8-4a9e-9d75-a22ac6ca8804 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:03.406658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:03.429987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca902b8-97a8-4a9e-9d75-a22ac6ca8804 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:03.472765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2a02-9fa7-441f-bac0-19cb4c4a7a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:03.473853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2a02-9fa7-441f-bac0-19cb4c4a7a31 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:03.474142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2a02-9fa7-441f-bac0-19cb4c4a7a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:03.474420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2a02-9fa7-441f-bac0-19cb4c4a7a31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:03.475731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2a02-9fa7-441f-bac0-19cb4c4a7a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:03.477262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2a02-9fa7-441f-bac0-19cb4c4a7a31 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:03.479005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 569/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:03.485387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:03.534080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:03.597540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35b43-d00a-4904-8291-ffcf834957eb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:03.633633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264ecf4-71de-481b-869f-47792c29e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:03.634308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264ecf4-71de-481b-869f-47792c29e0c9 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:03.634591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264ecf4-71de-481b-869f-47792c29e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:03.634871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264ecf4-71de-481b-869f-47792c29e0c9 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:03.635601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264ecf4-71de-481b-869f-47792c29e0c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:03.637087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264ecf4-71de-481b-869f-47792c29e0c9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:03.658429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 569/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:49:03 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:49:05.758374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac5c7-c042-4172-8a86-5d97cd21d279 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:05.758946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac5c7-c042-4172-8a86-5d97cd21d279 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:05.758946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac5c7-c042-4172-8a86-5d97cd21d279 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:05.760305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 570/1139] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:49:05.766498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:05.766946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:05.767135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:05.784013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:05.785274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:05.856474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 570/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:49:05.880967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ac5c7-c042-4172-8a86-5d97cd21d279 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:05.986599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ac5c7-c042-4172-8a86-5d97cd21d279 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:06.023325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:06.024035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:06.024317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:06.024624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:06.035457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:49:06.043165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:06.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:49:06.047166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 571/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:49:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:49:06.051681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:06.085328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:06.165269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88e650-9bb0-4411-9829-7f6e5d045b64 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:06.201537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:06.202215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:06.202498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:06.202774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:06.203532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:06.203839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:06.205256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:06.214066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 571/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:49:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:49:06.223815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:06.236264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:06.389048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4786-77f0-4fa8-afd2-5d710f4038e7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:06.448549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:06.450056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:06.450559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:06.451081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] PATH_INFO: `/v3/users/47fb6338319943abb0d0f74bd70a63fd` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:06.473822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] RBAC: Authorizing `identity:get_user(user_id=47fb6338319943abb0d0f74bd70a63fd)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:06.475364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:06.477243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 572/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:49:06 2022] GET /identity/v3/users/47fb6338319943abb0d0f74bd70a63fd => generated 286 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:06.483920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:06.575930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:06.589932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14984-f06e-4553-8352-b2a67c6d86d3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:06.624996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:06.625656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:06.625938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:06.626212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:06.704830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:06.706206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:06.802642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:06.816321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 572/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:49:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:49:06.826053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:06.925623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:06.967182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f879c7a-3d52-4409-886a-68637964d9d9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:07.007770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06f003-875a-400c-b88c-4dcca3bbbbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:07.008683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06f003-875a-400c-b88c-4dcca3bbbbcc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:07.008964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06f003-875a-400c-b88c-4dcca3bbbbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:07.009238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06f003-875a-400c-b88c-4dcca3bbbbcc None admin] PATH_INFO: `/v3/projects/4bd50cc319b3492fb1f03c735018dfdc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:07.010494 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06f003-875a-400c-b88c-4dcca3bbbbcc None admin] RBAC: Authorizing `identity:get_project(project_id=4bd50cc319b3492fb1f03c735018dfdc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:07.011942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06f003-875a-400c-b88c-4dcca3bbbbcc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:07.014288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 573/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:49:06 2022] GET /identity/v3/projects/4bd50cc319b3492fb1f03c735018dfdc => generated 311 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:07.024422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:07.194336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfca14b-dde7-49a7-afd7-b39c9f101220 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:07.251810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:07.252852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:07.253340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:07.253818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:07.255328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=4bd50cc319b3492fb1f03c735018dfdc, user.id=47fb6338319943abb0d0f74bd70a63fd) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:07.255857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:07.258645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0f37-e455-41b7-aa5c-01113da4c068 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:07.269466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 573/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:49:07 2022] GET /identity/v3/role_assignments?user.id=47fb6338319943abb0d0f74bd70a63fd&scope.project.id=4bd50cc319b3492fb1f03c735018dfdc&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:09.315331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9a8f7-ab3d-46b4-a975-62a0a41de38e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:09.315937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9a8f7-ab3d-46b4-a975-62a0a41de38e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:09.315937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9a8f7-ab3d-46b4-a975-62a0a41de38e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:09.317604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 574/1147] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:49:09.323950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:09.324538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:09.324538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:09.342794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:09.344060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:09.401164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:09.425658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 574/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:49:09.448081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9a8f7-ab3d-46b4-a975-62a0a41de38e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:09.572903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9a8f7-ab3d-46b4-a975-62a0a41de38e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:09.609788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:09.610615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:09.610902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:09.611183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:09.613684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:09.624905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:49:09.632642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:09.634486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:49:09.636160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 575/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:49:09.643558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:09.747339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590096b7-0900-4fcf-a7d4-e1c9fd4e55c3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:09.784338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:09.785035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:09.785460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:09.785621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:09.786415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:09.786722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:09.788178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:09.797400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 575/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:49:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:09.803462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:09.919744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb7066-9df6-4f3f-95fa-b45ce37cd033 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:09.957673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:09.958559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:09.958842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:09.959114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] PATH_INFO: `/v3/users/47fb6338319943abb0d0f74bd70a63fd` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:09.960343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] RBAC: Authorizing `identity:get_user(user_id=47fb6338319943abb0d0f74bd70a63fd)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:09.961837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:09.963543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 576/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:49:09 2022] GET /identity/v3/users/47fb6338319943abb0d0f74bd70a63fd => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:49:09.969552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:10.020607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:10.083953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e065167-6800-4e70-a515-2d65b7af0740 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:10.126179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d6be2-3367-4f01-9568-58817a0db488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:10.127166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6be2-3367-4f01-9568-58817a0db488 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:10.128017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6be2-3367-4f01-9568-58817a0db488 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:10.128362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d6be2-3367-4f01-9568-58817a0db488 None admin] PATH_INFO: `/v3/projects/4bd50cc319b3492fb1f03c735018dfdc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:10.130086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d6be2-3367-4f01-9568-58817a0db488 None admin] RBAC: Authorizing `identity:get_project(project_id=4bd50cc319b3492fb1f03c735018dfdc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:10.132733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d6be2-3367-4f01-9568-58817a0db488 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:10.134675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 576/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:49:09 2022] GET /identity/v3/projects/4bd50cc319b3492fb1f03c735018dfdc => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:10.142274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:10.245189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:10.313704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89d9f3-3db7-41c8-ba29-6567afe63318 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:10.360411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d969cd-3c8f-436e-bed9-4dd7e4800417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:10.361326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d969cd-3c8f-436e-bed9-4dd7e4800417 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:10.361610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d969cd-3c8f-436e-bed9-4dd7e4800417 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:10.361887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d969cd-3c8f-436e-bed9-4dd7e4800417 None admin] PATH_INFO: `/v3/projects/4bd50cc319b3492fb1f03c735018dfdc/users/47fb6338319943abb0d0f74bd70a63fd/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:10.363914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d969cd-3c8f-436e-bed9-4dd7e4800417 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bd50cc319b3492fb1f03c735018dfdc, user_id=47fb6338319943abb0d0f74bd70a63fd, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:10.365395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d969cd-3c8f-436e-bed9-4dd7e4800417 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:10.381099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 577/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:49:10 2022] PUT /identity/v3/projects/4bd50cc319b3492fb1f03c735018dfdc/users/47fb6338319943abb0d0f74bd70a63fd/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:49:12.439618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7fa62-c4c5-4c4f-b4a0-30be7479c5b9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:12.440235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7fa62-c4c5-4c4f-b4a0-30be7479c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:12.440235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7fa62-c4c5-4c4f-b4a0-30be7479c5b9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:12.441510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 577/1154] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:12.446999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:12.447499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:12.447628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:12.465628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:12.466842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:12.538669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 578/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:49:12.565219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7fa62-c4c5-4c4f-b4a0-30be7479c5b9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:12.669192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7fa62-c4c5-4c4f-b4a0-30be7479c5b9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:12.684058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:12.718020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:12.719119 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:12.719452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:12.719731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:12.738676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:49:12.746716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:12.748566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:49:12.749627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 578/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:49:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:49:12.755396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:12.828638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:12.882991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7dc96-09b9-44b6-93e1-a21c98bbf62e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:12.923918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:12.924621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:12.924901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:12.925172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:12.925919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:12.926221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:12.927610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:12.937305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 579/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:12.944435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:13.094400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45befbb-79f8-4667-b1c6-16c4abd09152 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:13.133819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:13.134728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:13.135014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:13.135297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] PATH_INFO: `/v3/users/47fb6338319943abb0d0f74bd70a63fd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:13.136693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] RBAC: Authorizing `identity:get_user(user_id=47fb6338319943abb0d0f74bd70a63fd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:13.138176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:13.140186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 579/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:49:12 2022] GET /identity/v3/users/47fb6338319943abb0d0f74bd70a63fd => generated 286 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:13.148053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:13.263625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661d408-4c6a-4bd6-8eb8-4e3b6542d5bd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:13.277981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:13.300170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:13.300851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:13.301131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:13.301405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:13.374743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:13.376124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:13.492323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 580/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:49:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:49:13.498548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:13.546099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:13.608009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339a285-fd87-492d-ba24-a9f2c2c04dbe None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:13.645450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d4504-cde0-4d00-a926-421d157e97aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:13.645913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4504-cde0-4d00-a926-421d157e97aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:13.646196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4504-cde0-4d00-a926-421d157e97aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:13.646470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4504-cde0-4d00-a926-421d157e97aa None admin] PATH_INFO: `/v3/projects/4bd50cc319b3492fb1f03c735018dfdc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:13.647686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d4504-cde0-4d00-a926-421d157e97aa None admin] RBAC: Authorizing `identity:get_project(project_id=4bd50cc319b3492fb1f03c735018dfdc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:13.649171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d4504-cde0-4d00-a926-421d157e97aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:13.650932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 580/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:49:13 2022] GET /identity/v3/projects/4bd50cc319b3492fb1f03c735018dfdc => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:49:13.657813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:13.673608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:13.761689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d27b3e-2911-487d-bcd7-3b6a8b77e7f7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:13.799945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:13.800921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:13.800921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:13.801217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:13.802082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=4bd50cc319b3492fb1f03c735018dfdc, user.id=47fb6338319943abb0d0f74bd70a63fd) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:13.802434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:13.803821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbfcb9-1b92-464f-92bc-ca0c698f802e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:13.810159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 581/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:49:13 2022] GET /identity/v3/role_assignments?user.id=47fb6338319943abb0d0f74bd70a63fd&scope.project.id=4bd50cc319b3492fb1f03c735018dfdc&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:15.892877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068b1dd-bf91-41c9-b8e7-d2fdffb82d21 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:15.893413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068b1dd-bf91-41c9-b8e7-d2fdffb82d21 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:15.893604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068b1dd-bf91-41c9-b8e7-d2fdffb82d21 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:15.895485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 581/1162] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:15.901342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:15.901821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:15.901941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:15.918817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:15.920573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:15.996976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 582/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:49:16.019413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068b1dd-bf91-41c9-b8e7-d2fdffb82d21 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:16.131559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068b1dd-bf91-41c9-b8e7-d2fdffb82d21 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:16.143243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:16.175820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:16.176604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:16.176931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:16.177257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:16.268112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:16.269605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:16.380228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 582/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:49:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:49:16.386202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:16.397157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:16.527004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1675e9-ac6b-45a7-81ec-e90589f8d695 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:16.569814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:16.570641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:16.570923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:16.571195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:16.579902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:16.581491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-66e3449a-db39-4c59-92f2-7c0e3ce28052 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:16.598577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:16.599914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 13:49:16.601072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:49:16.604848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 583/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:49:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:49:16.612865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:16.625516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:16.730813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340b78-8cf5-40c6-915b-613be3d4f2a6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:16.773180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:16.773856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:16.774186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:16.774452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:16.775225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:16.775522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:16.777002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2273c31-59f4-40e0-a6ab-bc388be11499 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:16.785149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 583/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jul 15 13:49:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:18.819137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e260b-618c-42a9-867a-f7d27e6d9e0f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:18.819737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e260b-618c-42a9-867a-f7d27e6d9e0f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:18.819737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e260b-618c-42a9-867a-f7d27e6d9e0f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:18.821105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 584/1167] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:18.826771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb30cb-eea9-4bb6-9f94-dd19f153d562 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:18.827273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb30cb-eea9-4bb6-9f94-dd19f153d562 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:18.827372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb30cb-eea9-4bb6-9f94-dd19f153d562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:18.844672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b8eb30cb-eea9-4bb6-9f94-dd19f153d562 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:18.845880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b8eb30cb-eea9-4bb6-9f94-dd19f153d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:18.922423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 584/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:49:18.946073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e260b-618c-42a9-867a-f7d27e6d9e0f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:19.023684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:19.066433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e260b-618c-42a9-867a-f7d27e6d9e0f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:19.126505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e69fb-2512-4f1a-9e0f-1ea7d66b6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:19.127510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e69fb-2512-4f1a-9e0f-1ea7d66b6c82 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:19.127970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e69fb-2512-4f1a-9e0f-1ea7d66b6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:19.128461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e69fb-2512-4f1a-9e0f-1ea7d66b6c82 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:19.129774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e69fb-2512-4f1a-9e0f-1ea7d66b6c82 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:19.132125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e69fb-2512-4f1a-9e0f-1ea7d66b6c82 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:19.152950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 585/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Fri Jul 15 13:49:18 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:49:21.224926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181e222-1f29-48a5-a7ab-a681a857c4e6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:21.225512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181e222-1f29-48a5-a7ab-a681a857c4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:21.225512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181e222-1f29-48a5-a7ab-a681a857c4e6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:21.226839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 585/1170] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:21.233675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:21.234197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:21.234409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:21.251928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:21.253189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:21.324009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 586/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:49:21.347842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181e222-1f29-48a5-a7ab-a681a857c4e6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:21.351037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:21.456128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181e222-1f29-48a5-a7ab-a681a857c4e6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:21.493550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b8df1-330c-48a4-9404-46aed2d446ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:21.494421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b8df1-330c-48a4-9404-46aed2d446ef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:21.494698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b8df1-330c-48a4-9404-46aed2d446ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:21.494971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b8df1-330c-48a4-9404-46aed2d446ef None admin] PATH_INFO: `/v3/domains/f5111b5ca0034f4aad8372c416fc718b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:21.506309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b8df1-330c-48a4-9404-46aed2d446ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5111b5ca0034f4aad8372c416fc718b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:21.507758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b8df1-330c-48a4-9404-46aed2d446ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:21.509516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 586/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Fri Jul 15 13:49:21 2022] GET /identity/v3/domains/f5111b5ca0034f4aad8372c416fc718b => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:21.515689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:21.581655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:21.628528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95a4b4-b654-4f5f-b069-47add2ba5805 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:21.667240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659827d4-1559-4b28-98e5-a8107867d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:21.667922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659827d4-1559-4b28-98e5-a8107867d3f8 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:21.668205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659827d4-1559-4b28-98e5-a8107867d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:21.668513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659827d4-1559-4b28-98e5-a8107867d3f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:21.669244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659827d4-1559-4b28-98e5-a8107867d3f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:21.670785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659827d4-1559-4b28-98e5-a8107867d3f8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:21.704841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 587/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:49:21 2022] POST /identity/v3/projects => generated 361 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:49:23.811622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba217d-576f-431b-a376-b1eeccc1a637 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:23.812161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba217d-576f-431b-a376-b1eeccc1a637 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:23.812161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba217d-576f-431b-a376-b1eeccc1a637 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:23.813610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 587/1174] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:23.820049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:23.820592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:23.820713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:23.838225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:23.839429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:23.912195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 588/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:49:23.938214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba217d-576f-431b-a376-b1eeccc1a637 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:24.004466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:24.044090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba217d-576f-431b-a376-b1eeccc1a637 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:24.080117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477313a-b308-48f2-828a-a5621036f7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:24.080930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477313a-b308-48f2-828a-a5621036f7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:24.081217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477313a-b308-48f2-828a-a5621036f7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:24.081494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477313a-b308-48f2-828a-a5621036f7f9 None admin] PATH_INFO: `/v3/domains/f5111b5ca0034f4aad8372c416fc718b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:24.082694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477313a-b308-48f2-828a-a5621036f7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5111b5ca0034f4aad8372c416fc718b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:24.084129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477313a-b308-48f2-828a-a5621036f7f9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:24.085825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 588/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Fri Jul 15 13:49:23 2022] GET /identity/v3/domains/f5111b5ca0034f4aad8372c416fc718b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:24.093027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:24.137136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:24.204558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04819b-5937-445c-ae26-8e174ac34c3b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:24.243928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287f431-bd93-4437-8e16-a07bb4d4a1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:24.244650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287f431-bd93-4437-8e16-a07bb4d4a1a4 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:24.244938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287f431-bd93-4437-8e16-a07bb4d4a1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:24.245217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287f431-bd93-4437-8e16-a07bb4d4a1a4 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:24.245961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287f431-bd93-4437-8e16-a07bb4d4a1a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:24.247434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287f431-bd93-4437-8e16-a07bb4d4a1a4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:24.268566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 589/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:49:24 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:49:26.364362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7091d2-0a1d-45e6-ae95-c3c9bff73081 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:26.364932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7091d2-0a1d-45e6-ae95-c3c9bff73081 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:26.364932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7091d2-0a1d-45e6-ae95-c3c9bff73081 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:26.366304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 589/1178] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:49:26.372016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:26.372508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:26.372622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:26.389999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:26.391206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:26.488828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 590/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:49:26.512750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7091d2-0a1d-45e6-ae95-c3c9bff73081 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:26.633807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7091d2-0a1d-45e6-ae95-c3c9bff73081 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:26.663973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:26.670331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:26.671133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:26.671419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:26.671701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:26.682284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:49:26.690550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:26.691749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:49:26.692735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 590/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:49:26.698692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:26.726518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:26.849721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc639fe-77c1-4f2a-8e14-a2dc6313ca2e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:26.887939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:26.888651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:26.888941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:26.889216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:26.889977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:26.890286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:26.891679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:26.901102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 591/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:26.907315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:27.015206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8a33f-d2aa-4591-9bb0-d017394796ab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:27.054443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:27.055313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:27.055590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:27.055862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] PATH_INFO: `/v3/users/e02ed804c8614b82846e28be15b3a7cb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:27.071450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] RBAC: Authorizing `identity:get_user(user_id=e02ed804c8614b82846e28be15b3a7cb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:27.072935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:27.074696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 591/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:49:26 2022] GET /identity/v3/users/e02ed804c8614b82846e28be15b3a7cb => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:49:27.083719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:27.179834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:27.193292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e2df-b442-46a4-a2b5-bf10a506f76f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:27.229218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:27.229871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:27.230150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:27.230424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:27.301782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:27.303819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:27.439657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:27.454590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 592/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:49:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:49:27.461221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:27.521446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:27.575120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3d849-3a2c-4128-b79f-bd6906f241f4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:27.615395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f08615-1e12-47d7-9fcf-818f4b85e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:27.616260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08615-1e12-47d7-9fcf-818f4b85e22b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:27.616573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08615-1e12-47d7-9fcf-818f4b85e22b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:27.616848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f08615-1e12-47d7-9fcf-818f4b85e22b None admin] PATH_INFO: `/v3/projects/9422518d6c84484eb109d60690d4bf49` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:27.618072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f08615-1e12-47d7-9fcf-818f4b85e22b None admin] RBAC: Authorizing `identity:get_project(project_id=9422518d6c84484eb109d60690d4bf49)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:27.619512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f08615-1e12-47d7-9fcf-818f4b85e22b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:27.621279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 592/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:49:27 2022] GET /identity/v3/projects/9422518d6c84484eb109d60690d4bf49 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:27.627555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:27.733961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ed9ff-3483-4fef-921e-d819802cfba6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:27.775601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:27.776279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:27.776592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:27.776880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:27.777736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=9422518d6c84484eb109d60690d4bf49, user.id=e02ed804c8614b82846e28be15b3a7cb) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:27.778052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:27.780602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f471-e592-4834-b1d1-817835935eb2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:27.787644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 593/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:49:27 2022] GET /identity/v3/role_assignments?user.id=e02ed804c8614b82846e28be15b3a7cb&scope.project.id=9422518d6c84484eb109d60690d4bf49&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:49:29.809399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c59cb-0f38-4871-a69f-c31a95276752 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:29.810501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c59cb-0f38-4871-a69f-c31a95276752 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:29.810501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c59cb-0f38-4871-a69f-c31a95276752 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:29.811952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 593/1186] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:49:29.818438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:29.818904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:29.819009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:29.836069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:29.837308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:29.886819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:29.909266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 594/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:49:29.931772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c59cb-0f38-4871-a69f-c31a95276752 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:30.036528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c59cb-0f38-4871-a69f-c31a95276752 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:30.053972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:30.073351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:30.074067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:30.074341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:30.074612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:30.086381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:49:30.094292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:30.095778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:49:30.097226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 594/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:49:30.102035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:30.208778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0c975-839a-4194-bcd8-a7891bf07a10 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:30.246882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:30.247594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:30.247878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:30.248154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:30.248946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:30.249253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:30.250630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:30.262290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 595/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:49:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:30.267479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:30.371587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2b62a-1ba6-4fd3-acb9-a2fda028f573 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:30.409274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:30.410145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:30.410427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:30.410703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] PATH_INFO: `/v3/users/e02ed804c8614b82846e28be15b3a7cb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:30.411911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=e02ed804c8614b82846e28be15b3a7cb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:30.413391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:30.415055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 595/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:49:30 2022] GET /identity/v3/users/e02ed804c8614b82846e28be15b3a7cb => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:30.421530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:30.469507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:30.536248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f7a6e-c37b-4842-ad0a-ba1bf44247cf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:30.570382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32689dae-6a60-439a-834c-8f549d76fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:30.571246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32689dae-6a60-439a-834c-8f549d76fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:30.571529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32689dae-6a60-439a-834c-8f549d76fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:30.571805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32689dae-6a60-439a-834c-8f549d76fdb3 None admin] PATH_INFO: `/v3/projects/9422518d6c84484eb109d60690d4bf49` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:30.573023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32689dae-6a60-439a-834c-8f549d76fdb3 None admin] RBAC: Authorizing `identity:get_project(project_id=9422518d6c84484eb109d60690d4bf49)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:30.574469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32689dae-6a60-439a-834c-8f549d76fdb3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:30.576725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 596/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:49:30 2022] GET /identity/v3/projects/9422518d6c84484eb109d60690d4bf49 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:30.582971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:30.631475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:30.690399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e21d0-2ac2-40be-a1b6-02ce2abd1ec7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:30.725612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b893b-1610-47ac-8b00-62b1dc0d0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:30.726509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b893b-1610-47ac-8b00-62b1dc0d0979 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:30.726794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b893b-1610-47ac-8b00-62b1dc0d0979 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:30.727073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b893b-1610-47ac-8b00-62b1dc0d0979 None admin] PATH_INFO: `/v3/projects/9422518d6c84484eb109d60690d4bf49/users/e02ed804c8614b82846e28be15b3a7cb/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:30.729100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b893b-1610-47ac-8b00-62b1dc0d0979 None admin] RBAC: Authorizing `identity:create_grant(project_id=9422518d6c84484eb109d60690d4bf49, user_id=e02ed804c8614b82846e28be15b3a7cb, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:30.730558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b893b-1610-47ac-8b00-62b1dc0d0979 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:30.745242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 596/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:49:30 2022] PUT /identity/v3/projects/9422518d6c84484eb109d60690d4bf49/users/e02ed804c8614b82846e28be15b3a7cb/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:49:32.794863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aae20-5943-4604-850f-8a374e9ca2f3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:32.795503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aae20-5943-4604-850f-8a374e9ca2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:32.795503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aae20-5943-4604-850f-8a374e9ca2f3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:32.796843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 597/1193] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:49:32.802153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:32.802625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:32.802724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:32.819670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:49:32.820903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:49:32.890185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 597/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:49:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 13:49:32.913859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aae20-5943-4604-850f-8a374e9ca2f3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:33.024214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aae20-5943-4604-850f-8a374e9ca2f3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:33.038150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:33.060571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:33.061365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:33.061648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:33.061921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:33.072912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:49:33.082026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:49:33.083162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:49:33.084297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 598/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:49:33.089467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:33.123463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:33.198922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3737e5-609a-46eb-b4fd-7927046a557b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:33.234487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:33.235160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:33.235436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:33.235711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:33.236495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:33.236797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:33.238170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:33.246620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 598/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:49:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:33.253154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:33.358034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05391a1-0a61-4915-8c62-1900251876f9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:33.393121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:33.393992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:33.394271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:33.394548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] PATH_INFO: `/v3/users/e02ed804c8614b82846e28be15b3a7cb` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:33.395761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] RBAC: Authorizing `identity:get_user(user_id=e02ed804c8614b82846e28be15b3a7cb)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:33.397227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:33.398890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 599/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jul 15 13:49:33 2022] GET /identity/v3/users/e02ed804c8614b82846e28be15b3a7cb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:33.404853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:33.508707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:33.515657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94e7c-a49b-41e2-b44f-cf155cac14e5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:33.551169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:33.551824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:33.552109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:33.552414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:33.622912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:33.624290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:33.717730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:33.725001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 599/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:49:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:49:33.731119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:33.778312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:33.839765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebae82-9b2f-4b19-8842-61583c2451e8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:33.874436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaadeee-55ab-4e38-ad1f-7e84910fe304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:33.875223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaadeee-55ab-4e38-ad1f-7e84910fe304 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:33.875504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaadeee-55ab-4e38-ad1f-7e84910fe304 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:33.875783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaadeee-55ab-4e38-ad1f-7e84910fe304 None admin] PATH_INFO: `/v3/projects/9422518d6c84484eb109d60690d4bf49` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:33.877041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaadeee-55ab-4e38-ad1f-7e84910fe304 None admin] RBAC: Authorizing `identity:get_project(project_id=9422518d6c84484eb109d60690d4bf49)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:33.878522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaadeee-55ab-4e38-ad1f-7e84910fe304 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:33.880296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 600/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jul 15 13:49:33 2022] GET /identity/v3/projects/9422518d6c84484eb109d60690d4bf49 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:49:33.886708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:49:33.988590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6339a-78b1-4ec7-916f-03560842ecb4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:49:34.022051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:49:34.022695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:49:34.022977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:49:34.023243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:49:34.024064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=9422518d6c84484eb109d60690d4bf49, user.id=e02ed804c8614b82846e28be15b3a7cb) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:49:34.024364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:49:34.025808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f58aa9-dc6b-4d47-9ac3-310c0fb96437 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:49:34.032519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 600/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:49:33 2022] GET /identity/v3/role_assignments?user.id=e02ed804c8614b82846e28be15b3a7cb&scope.project.id=9422518d6c84484eb109d60690d4bf49&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:49:43.870042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=46 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:49:44.029993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=70 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:30.209027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201555-254d-44fc-b98a-9376e4c12cce None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:30.209893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201555-254d-44fc-b98a-9376e4c12cce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:30.209893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201555-254d-44fc-b98a-9376e4c12cce None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:30.212058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 601/1201] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:30.222084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:30.223957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:30.224482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:30.253867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:30.257723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:30.374144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 601/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:30.400985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201555-254d-44fc-b98a-9376e4c12cce None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:30.578332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201555-254d-44fc-b98a-9376e4c12cce None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:30.638687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:30.639532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:30.639838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:30.640099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:30.654866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:30.656321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:30.677636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:30.678645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:30.679608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:30.679608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:30.679608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:30.679608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:30.686330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:30.688085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 602/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:30.822582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98d73b-ffb8-40d2-8d70-cbe4f1049023 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:30.861835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:30.868434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:30.869108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:30.869437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:30.869809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:30.870656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:30.871599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:30.873387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13712b-8df9-4388-ba2b-7ba092bf0b19 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:30.892495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 602/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:30.895123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:30.966860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:31.043836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcafdf-23c8-4756-8513-aa1f05d57bbe None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:31.096494 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a90975-150e-42d6-be62-fa914d1c08d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:31.097740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a90975-150e-42d6-be62-fa914d1c08d0 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:31.098066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a90975-150e-42d6-be62-fa914d1c08d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:31.098295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a90975-150e-42d6-be62-fa914d1c08d0 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:31.099399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a90975-150e-42d6-be62-fa914d1c08d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:31.100958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a90975-150e-42d6-be62-fa914d1c08d0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:31.130804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 603/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jul 15 13:50:30 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:50:33.381927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662cf67-7d8b-4a7c-a7b6-f029bed11dc8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:33.382476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662cf67-7d8b-4a7c-a7b6-f029bed11dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:33.382476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662cf67-7d8b-4a7c-a7b6-f029bed11dc8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:33.384111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 603/1206] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:33.391285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:33.391862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:33.391956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:33.413800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:33.415456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:33.509847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 604/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:33.537120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662cf67-7d8b-4a7c-a7b6-f029bed11dc8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:33.680305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662cf67-7d8b-4a7c-a7b6-f029bed11dc8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:33.728381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:33.729363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:33.729642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:33.729935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:33.741927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:50:33.751536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:33.752565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:33.753507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 604/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:50:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:33.759391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:33.807676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:33.896100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23058d6-47a2-4119-9b68-04d10d8bbffd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:33.941734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:33.942764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:33.943062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:33.943340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:33.944170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:33.944522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:33.948529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:33.958203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 605/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:50:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:33.964147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:34.031482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:34.087387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fe496-a0aa-40e9-a0c2-25a9623d0e63 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:34.127157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:34.127965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:34.128249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:34.128557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:34.140639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:34.142060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:34.160745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:34.161873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:34.162788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:34.162788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:34.162788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:34.162788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:34.162788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 605/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:34.167669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:34.286033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:34.289184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a645-6636-46cc-b686-0a8d3a305f71 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:34.330276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:34.330942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:34.331225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:34.331527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:34.332261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:34.332587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:34.333977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:34.347531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 606/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:34.353625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:34.478658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e5628-83a6-4562-b199-c9ae95242bff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:34.480595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:34.520014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:34.520857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:34.521205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:34.521478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:34.530109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:34.531695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:50:34.548045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:34.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:34.550458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 606/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:34.557236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:34.670762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9176e-a20d-4aa8-b2e7-bddf7eec5321 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:34.713709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:34.714389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:34.714715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:34.714996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:34.715839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:34.716154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:34.717608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:34.736745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 607/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jul 15 13:50:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:34.742580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:34.875724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df134b2-aa17-4b3f-9ae5-3e4322a8b6fc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:34.923137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:34.923882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:34.924238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:34.924608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:34.972185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:35.022593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:35.024115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:35.186868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 607/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:50:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:50:35.187068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:35.213730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:35.327435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c0ea-7fd6-4e46-a007-d7671657f455 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:35.369455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:35.370275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:35.370559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:35.370934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:35.383940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:35.385511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:35.413013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:35.414307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:35.420469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:35.420469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:35.420469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:35.420469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:35.420469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 608/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:35.420781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:35.493666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:35.538895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28cff3-0eb4-42e1-ac92-ec400bfedaeb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:35.581258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:35.582045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:35.582358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:35.582767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:35.583511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:35.583843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:35.585404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:35.602544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 608/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:35.603752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:35.691696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:35.729676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe9cca-b5a0-4d07-a21c-18c95cfb5adf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:35.771386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:35.772197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:35.772516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:35.772900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:35.780870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:35.782357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:50:35.798277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:35.799762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:35.803864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 609/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:35.806644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:35.925291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a251d4-8c95-4eb1-9132-902d6d8089a3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:35.955871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:35.972028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:35.972745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:35.973025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:35.973300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:35.974393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:35.974725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:35.976169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6582df1-188d-4208-bf06-ef4a253cd2ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:35.990766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 609/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:35.997378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:36.112820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20659ced-e3ac-4c5d-b97c-416b8d6a392b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:36.146055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:36.152612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:36.153360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:36.153666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:36.153952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:36.154833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=92ac8ddf274a45d1b56533944a2334fb) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:36.155174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:36.156637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0961ed-ce79-46c3-a260-9b665d797a79 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:36.163618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 610/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:50:35 2022] GET /identity/v3/role_assignments?user.id=92ac8ddf274a45d1b56533944a2334fb&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:38.207665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07022824-78b5-4671-ad5b-a04950ffcc4e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:38.208279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07022824-78b5-4671-ad5b-a04950ffcc4e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:38.208279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07022824-78b5-4671-ad5b-a04950ffcc4e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:38.209631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 610/1220] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:38.215797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:38.216280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:38.216434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:38.235239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:38.236544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:38.309154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 611/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:38.332919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07022824-78b5-4671-ad5b-a04950ffcc4e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:38.450371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07022824-78b5-4671-ad5b-a04950ffcc4e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:38.489227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:38.490319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:38.490601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:38.490876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:38.502277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:50:38.509984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:38.511060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:38.512878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 611/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:50:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:38.517911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:38.627495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95db22a-48ed-47b1-9891-f5b0ce5b0cc9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:38.654041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:38.665058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:38.666019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:38.666499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:38.666937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:38.667732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:38.668083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:38.669583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:38.683088 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 612/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:38.684902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:38.731202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:38.791641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77a53a-dc8e-4c83-bb5e-e9442fe273e8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:38.831824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:38.832674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:38.832943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:38.833215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:38.848911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:38.850381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:38.868974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:38.870286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:38.871889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:38.871889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:38.871889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:38.871889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:38.871889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 612/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:38.876842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:38.986311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c005d-0c65-40f9-9c8f-e518e5cc0ae5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:39.022160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:39.022831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:39.023116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:39.023394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:39.024122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:39.024456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:39.025837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:39.038177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 613/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:39.045035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:39.155871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:39.163601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb37fad-412a-417d-b748-332a19fa118c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:39.206706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:39.207509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:39.207791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:39.208417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:39.215901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:39.217376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:50:39.232825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:39.234296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:39.235299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 613/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:50:39.240136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:39.289748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:39.344739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028768c9-1194-4bac-963a-ea3b636fb40a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:39.388826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:39.389639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:39.389942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:39.390281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:39.391411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:39.391780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:39.393623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:39.418673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 614/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jul 15 13:50:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:39.424378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:39.540624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a54d3-80d5-49bb-920c-0895c7fe42b3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:39.580998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:39.581924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:39.582210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:39.582487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:39.595321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:39.596872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:39.617610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:39.618644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:39.619972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:39.619972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:39.619972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:39.619972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:39.619972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 614/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:39.625455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:39.743323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9dd58-3d2b-4f45-a8c2-8aa71728b0cc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:39.745725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:39.781597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:39.782336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:39.782621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:39.783009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:39.783816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:39.784149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:39.785622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:39.798456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 615/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:39.804541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:39.813914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:39.912598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892529-455c-454c-ba01-6604ec16dd54 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:39.951509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:39.952301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:39.952612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:39.952887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:39.960008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:39.961469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:50:39.975983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:39.977184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:39.978294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 615/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:50:39.983249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:40.092922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf30d-359c-4053-81e9-2f141293e986 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:40.132061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:40.132792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:40.133060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:40.133335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:40.134206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:40.134514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:40.136038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06728b-515f-48f9-abd8-8c0e51065899 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:40.150587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 616/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:40.157054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:40.231751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:40.270660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7128c2e-0cb3-4b01-afc0-6f6a75573765 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:40.310665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90ff72-ef24-4d06-98e4-8a951fa9f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:40.311577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90ff72-ef24-4d06-98e4-8a951fa9f0d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:40.311858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90ff72-ef24-4d06-98e4-8a951fa9f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:40.312132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90ff72-ef24-4d06-98e4-8a951fa9f0d2 None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/92ac8ddf274a45d1b56533944a2334fb/roles/e2f5f87ca4c2442d99da472566d384a5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:40.328830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90ff72-ef24-4d06-98e4-8a951fa9f0d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=92ac8ddf274a45d1b56533944a2334fb, role_id=e2f5f87ca4c2442d99da472566d384a5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:40.330325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90ff72-ef24-4d06-98e4-8a951fa9f0d2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:40.344281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 616/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:50:40 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/92ac8ddf274a45d1b56533944a2334fb/roles/e2f5f87ca4c2442d99da472566d384a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:50:42.576583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c544fa-e1d6-4174-902d-72ffe406c9c9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:42.577124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c544fa-e1d6-4174-902d-72ffe406c9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:42.577124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c544fa-e1d6-4174-902d-72ffe406c9c9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:42.578522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 617/1233] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:42.583779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:42.584255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:42.584364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:42.602187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:42.603403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:42.677512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 617/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:42.702547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c544fa-e1d6-4174-902d-72ffe406c9c9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:42.729232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:42.810445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c544fa-e1d6-4174-902d-72ffe406c9c9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:42.845634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:42.846353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:42.846635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:42.846908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:42.858567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:50:42.866395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:42.867544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:42.868983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 618/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jul 15 13:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:42.874532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:42.931169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:42.996059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d698b5-68be-4b68-b5d6-d33a4c8f836e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:43.037395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:43.038593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:43.038935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:43.039237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:43.040261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:43.040645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:43.042161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:43.052327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 618/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jul 15 13:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:43.058404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:43.157337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:43.176322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f60f86-9895-40df-b862-f67ca4886585 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:43.217634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:43.218441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:43.218725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:43.219003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:43.231646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:43.233189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:43.252133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:43.253152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:43.254030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:43.254030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:43.254030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:43.254030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:43.254030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 619/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:43.260778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:43.366727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954bcf-ce9e-4afb-81d9-ab7fdce38788 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:43.376591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:43.403918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:43.404610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:43.404889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:43.405158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:43.405872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:43.406172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:43.407720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:43.421217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 619/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:43.427099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:43.533879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502199-68b0-4a96-a995-44615a8aa08a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:43.574626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:43.575729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:43.576054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:43.576337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:43.580444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:43.587334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:43.589259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:50:43.606331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:43.608141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:43.609122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 620/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:43.614099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:43.730321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd41a-b85d-421b-ad69-214cf38c53df None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:43.777196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:43.777196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:43.777196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:43.777196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:43.777763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:43.777763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:43.777763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:43.798595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 620/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jul 15 13:50:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:43.805525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:43.923125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad8354-dee4-4d44-b340-7d0f291b21fe None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:43.969199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:43.969865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:43.970225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:43.970432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:44.047275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:44.055222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:44.056814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:44.174053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 621/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:50:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 13:50:44.180179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:44.212710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:44.291959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3ea30-59c8-4e28-9785-bde19db7e6d3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:44.328168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:44.328998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:44.329263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:44.329571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:44.340271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:44.341853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:44.359854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:44.361013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:44.362478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:44.362478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:44.362478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:44.362478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:44.362478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 621/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:44.367323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:44.478986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:44.485268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9d557-1ffc-4e7f-9e44-4c7083b572ac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:44.527966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:44.528807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:44.529090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:44.529478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:44.530343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:44.530659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:44.532042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:44.546871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 622/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:44.554331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:44.641780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:44.668298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed44f-c7a1-422c-a7b4-f84f8e790a06 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:44.707431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:44.708346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:44.708812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:44.709088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:44.718290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:44.720208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:50:44.738492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:44.739567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:44.740477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 622/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:44.747525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:44.858772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3674c1-ee70-420c-9658-fc39d460cc66 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:44.896605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:44.897271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:44.897542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:44.897790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:44.898579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:44.898901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:44.900304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd2b02-4d23-418b-a63c-3035f987dd21 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:44.913923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 623/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:44.920634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:45.043081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4897f-01d2-430b-9715-1d3f320feba8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:45.078161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:45.078836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:45.079248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:45.079540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:45.080440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] RBAC: Adding query filter params (role.id=e2f5f87ca4c2442d99da472566d384a5, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=92ac8ddf274a45d1b56533944a2334fb) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:45.080755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:45.082185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb0ae2-aca0-44d3-aa89-a5ef58e7839e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:45.090070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 623/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:50:44 2022] GET /identity/v3/role_assignments?user.id=92ac8ddf274a45d1b56533944a2334fb&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=e2f5f87ca4c2442d99da472566d384a5 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:47.358081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3f42-91d0-4c83-8739-8d5096380fb3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:47.358655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3f42-91d0-4c83-8739-8d5096380fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:47.358655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb3f42-91d0-4c83-8739-8d5096380fb3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:47.360189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 624/1247] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:47.366606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:47.367262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:47.367262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:47.373660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:47.389329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:47.390813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:47.469555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 624/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:47.497128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb3f42-91d0-4c83-8739-8d5096380fb3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:47.519301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:47.625093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb3f42-91d0-4c83-8739-8d5096380fb3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:47.670823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:47.672177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:47.672706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:47.673192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:47.690647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:50:47.701042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:47.702246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:47.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 625/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:47.708901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:47.836107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeebbef-2f06-4d98-ad91-51386340b7aa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:47.867490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:47.874399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:47.875235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:47.875579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:47.875920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:47.880413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:47.880413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:47.880413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:47.889752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 625/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:47.895671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:47.994785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:48.027404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964711b0-3309-4ecf-9d3b-f0b66627d359 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:48.066962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:48.067759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:48.068065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:48.068447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:48.081119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:48.082610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:48.102372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:48.103395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:48.104285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:48.104285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:48.104285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:48.104285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:48.104285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 626/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:50:48.109797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:48.232227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cb5e7-be9c-47fe-acae-95a1635a104a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:48.275925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:48.278263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:48.278520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:48.278833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:48.279594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:48.279902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:48.281343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:48.295381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 626/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:48.301371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:48.415391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc09e-6b73-407e-af3a-c78c01e69420 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:48.448203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:48.459038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be765-af00-4514-b05b-45a2251216da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:48.459927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be765-af00-4514-b05b-45a2251216da None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:48.460277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be765-af00-4514-b05b-45a2251216da None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:48.460632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be765-af00-4514-b05b-45a2251216da None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:48.470440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401be765-af00-4514-b05b-45a2251216da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:48.472258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401be765-af00-4514-b05b-45a2251216da None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-401be765-af00-4514-b05b-45a2251216da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:50:48.491545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:48.492846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:48.493981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 627/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:48.499517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:48.568132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:48.622956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ca0cf-8715-47f3-a054-8f2de9ced725 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:48.660413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:48.661126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:48.661407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:48.661682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:48.662596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:48.662904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:48.664383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:48.685694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 627/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jul 15 13:50:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:48.692051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be765-af00-4514-b05b-45a2251216da None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:48.811502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be765-af00-4514-b05b-45a2251216da None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:48.851109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:48.851787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:48.852065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:48.852341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:48.920781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:48.938207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:48.939748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:49.073202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 628/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:50:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:50:49.075436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:49.197087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ad234-2327-4da1-9713-bf7b98c309a1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:49.211832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:49.234341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:49.235208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:49.235489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:49.235792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:49.247801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:49.249375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:49.272741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:49.273989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:49.275372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:49.275372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:49.275372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:49.275372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:49.275372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 628/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:49.280278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:49.379620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:49.394158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9e7a-3df6-41b9-a007-d3f339841275 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:49.432128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:49.432819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:49.433167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:49.434030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:49.434718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:49.435092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:49.436506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:49.454297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 629/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:49.455527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:49.559754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155e7b5-45fb-4e4b-bee5-d1d109b09ce1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:49.598020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:49.599096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:49.599380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:49.599660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:49.608126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:49.609635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:50:49.625722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:49.626933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:49.627904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 629/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:49.633200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:49.746458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849089e-c4e0-41c9-a834-330d662a3209 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:49.783021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:49.783678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:49.783963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:49.784241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:49.785068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:49.785384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:49.786829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78807f92-9077-4e50-9807-ec2d9e715eb8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:49.800077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 630/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:49.806292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:49.836663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:49.920340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4543fb-a843-454b-87a4-1b8bf1cecca2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:49.965291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:49.966040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:49.966327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:49.966656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:49.967818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=92ac8ddf274a45d1b56533944a2334fb) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:49.968182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:49.970041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535de19-32cb-4b6f-a4c4-af2040623827 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:49.978062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 630/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:50:49 2022] GET /identity/v3/role_assignments?user.id=92ac8ddf274a45d1b56533944a2334fb&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=10895c39249448e4ac3616747cafa76b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:51.979687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b1f8-860c-49c7-a777-9576237e272f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:51.980834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b1f8-860c-49c7-a777-9576237e272f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:51.980834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b1f8-860c-49c7-a777-9576237e272f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:51.982206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 631/1261] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:51.987736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112116df-93cd-4456-ac51-df0f14330637 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:51.988170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112116df-93cd-4456-ac51-df0f14330637 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:51.988269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112116df-93cd-4456-ac51-df0f14330637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:52.004751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-112116df-93cd-4456-ac51-df0f14330637 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:52.005926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-112116df-93cd-4456-ac51-df0f14330637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:52.070480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 631/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:52.095033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b1f8-860c-49c7-a777-9576237e272f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:52.106872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:52.197650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b1f8-860c-49c7-a777-9576237e272f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:52.234153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:52.234852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:52.235132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:52.235409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:52.245392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:50:52.252983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:52.254098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:52.255018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 632/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:52.260305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112116df-93cd-4456-ac51-df0f14330637 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:52.279036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:52.365875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112116df-93cd-4456-ac51-df0f14330637 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:52.404551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:52.405218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:52.405499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:52.405779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:52.406521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:52.406824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:52.408182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:52.416206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 632/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:50:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:52.421956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:52.487296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:52.520864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b3455-768a-40e0-a213-131e04071719 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:52.553781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:52.554565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:52.554845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:52.555119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:52.566108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:52.567541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:52.584087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:52.585250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:52.586289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:52.586289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:52.586289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:52.586289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:52.586289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 633/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:52.591128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:52.671960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:52.691587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304b07e-0cfb-4f5a-8536-90da5382d2bc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:52.724108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:52.724780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:52.725062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:52.725334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:52.726046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:52.726348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:52.727704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:52.739805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 633/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:52.745428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:52.841563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c22cf6-54d4-4305-badc-0f734824efe4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:52.861483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:52.875569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:52.876376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:52.876686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:52.876960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:52.886626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:52.888104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:50:52.904076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:52.905333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:52.906314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 634/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:50:52.911887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:53.021568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5cd66-f189-44de-937e-1dedbcc48b6b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:53.055618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:53.056292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:53.056674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:53.056954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:53.057789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:53.058126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:53.059558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:53.068205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:53.078246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 634/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jul 15 13:50:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:53.084148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:53.185068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75727528-30e7-43c7-a78c-d2a57ebd37cf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:53.218552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:53.219335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:53.219626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:53.219906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:53.230661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:53.232100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:53.248685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:53.249723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:53.250644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:53.250644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:53.250644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:53.250644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:53.250644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 635/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:53.255845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:53.365882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee69f2-cd12-49fd-a079-5884c9c8fa85 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:53.402876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:53.403545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:53.403828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:53.404103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:53.404879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:53.405187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:53.406593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:53.419533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 635/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:53.425460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:53.457997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:53.528415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fcf64-8ce2-470b-8b8f-1786422d5dc0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:53.561917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:53.562713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:53.563001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:53.563281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:53.569954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:53.571399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:50:53.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:53.587278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:53.588489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 636/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:53.594561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:53.652531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:53.694275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6b5ab-8c45-454e-9810-3cfeb3f9e41c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:53.727941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:53.728696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:53.728984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:53.729260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:53.730046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:53.730351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:53.731780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcfe93-da83-4869-abf1-b00f6bc946a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:53.743828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 636/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:53.750411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:53.840725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:53.849786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcc1f1-a216-4c9f-a945-530eb61720d2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:53.888225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e78b5-0872-4a18-8528-fc7ddc26e84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:53.889928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e78b5-0872-4a18-8528-fc7ddc26e84b None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:53.890460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e78b5-0872-4a18-8528-fc7ddc26e84b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:53.890943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e78b5-0872-4a18-8528-fc7ddc26e84b None admin] PATH_INFO: `/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/92ac8ddf274a45d1b56533944a2334fb/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:53.893370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e78b5-0872-4a18-8528-fc7ddc26e84b None admin] RBAC: Authorizing `identity:create_grant(project_id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user_id=92ac8ddf274a45d1b56533944a2334fb, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:53.894808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e78b5-0872-4a18-8528-fc7ddc26e84b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:53.907430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 637/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jul 15 13:50:53 2022] PUT /identity/v3/projects/3daa91b4a48b4ed0bc5f2cc3b87094ae/users/92ac8ddf274a45d1b56533944a2334fb/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:50:55.929841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f9e60-1332-4cc9-bebd-ee5afa4d3a4c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:55.930521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f9e60-1332-4cc9-bebd-ee5afa4d3a4c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:55.930521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f9e60-1332-4cc9-bebd-ee5afa4d3a4c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:55.932185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 637/1274] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:50:55.939081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:55.939644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:55.939936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:55.958148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:50:55.959339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:50:56.024486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 638/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:50:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:50:56.048710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f9e60-1332-4cc9-bebd-ee5afa4d3a4c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:56.154717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f9e60-1332-4cc9-bebd-ee5afa4d3a4c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:56.171947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:56.188962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:56.189685 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:56.189970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:56.190248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:56.200744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 13:50:56.208235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:56.209353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:56.210536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 638/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jul 15 13:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:56.216082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:56.301840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:56.321342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964bab8-e021-4d52-a2bf-ce8df214ee29 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:56.359217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:56.359892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:56.360171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:56.360470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:56.361227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:56.361527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:56.362917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:56.372013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 639/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jul 15 13:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:56.377944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:56.490298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55761f1-0f2f-4a80-98ce-c5d6df12d357 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:56.526905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:56.527705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:56.527987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:56.528265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:56.539797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:56.541535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:56.559758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:56.560875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:56.561987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:56.561987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:56.561987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:56.561987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:56.561987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 639/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:56.566816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:56.670985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c40fb2-81f9-422c-94a9-ffbc58c2179a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:56.705985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:56.706677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:56.706961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:56.707237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:56.707965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:56.708268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:56.709666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:56.721645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 640/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:56.727673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:56.758716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:56.830748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f4e38-f632-43a2-bd3d-879e6a303c67 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:56.864445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:56.865242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:56.865528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:56.865799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:56.874195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:56.875644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 13:50:56.890266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:56.891410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:50:56.892755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 640/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:56.897362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:56.901211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:57.020877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de1d61-19ae-4ca6-8cb5-5bd04ef68f20 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:57.056145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:57.056860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:57.057146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:57.057424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:57.058839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:57.059143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:57.060746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:57.078424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 641/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jul 15 13:50:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:57.084172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:57.179627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:57.193963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2ad1a-46a8-44ce-9a9e-bda28df56de5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:57.231228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:57.231876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:57.232161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:57.232463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:57.300938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:57.302328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:57.397082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:57.408270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 641/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jul 15 13:50:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 13:50:57.414178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:57.479067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:57.519677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d86da-0022-46dd-8f02-3ce2acc244ca None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:57.554974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:57.555773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:57.556057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:57.556331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:57.566571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:57.568032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 13:50:57.590396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:57.591623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:57.592611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 13:50:57.592611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 13:50:57.592611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 13:50:57.592611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:57.592611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 642/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:57.597439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:57.700081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8bc7f-b2db-4cd8-9fbb-6f95b96ac649 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:57.735705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:57.736884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:57.738721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:57.738721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:57.738721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:57.738721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:57.740064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:57.754543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 642/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jul 15 13:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 13:50:57.760505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:57.859474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205133e-7c31-4402-8333-65f7a498d31f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:57.873989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:57.894765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:57.895552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:57.895831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:57.896117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:57.903409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:57.904871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 13:50:57.919767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:50:57.922081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 15 13:50:57.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 643/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jul 15 13:50:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:50:57.927023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:57.981110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:50:58.032673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785dbc1-f263-445b-8822-b2c084868e7b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:58.066576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:58.067236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:58.067520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:58.067796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:58.068607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:58.068915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:58.070359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57da1b-0e7a-48b4-8c39-7bbcb0705493 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:58.083238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 643/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jul 15 13:50:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:50:58.089399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:50:58.198029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c882e-9c8a-43eb-8147-b2d74332f163 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:50:58.232361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:50:58.233059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:50:58.233346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:50:58.233623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:50:58.234453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] RBAC: Adding query filter params (role.id=10895c39249448e4ac3616747cafa76b, scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae, user.id=92ac8ddf274a45d1b56533944a2334fb) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:50:58.234790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:50:58.236221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b61c4-a968-4674-8499-41ca87850a59 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:50:58.242311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 644/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jul 15 13:50:58 2022] GET /identity/v3/role_assignments?user.id=92ac8ddf274a45d1b56533944a2334fb&scope.project.id=3daa91b4a48b4ed0bc5f2cc3b87094ae&role.id=10895c39249448e4ac3616747cafa76b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:51:00.278366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782dcdd-a9d0-4ea9-bcb0-e840ebe9d406 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:00.278886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782dcdd-a9d0-4ea9-bcb0-e840ebe9d406 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:00.279001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782dcdd-a9d0-4ea9-bcb0-e840ebe9d406 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:00.280301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 644/1288] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:51:00.285398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:00.285773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:00.285957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:00.303072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:51:00.304266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:51:00.332936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:00.371117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 645/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:51:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:51:00.392959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782dcdd-a9d0-4ea9-bcb0-e840ebe9d406 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:00.496790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782dcdd-a9d0-4ea9-bcb0-e840ebe9d406 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:00.508810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:00.532489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:00.533278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:00.533561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:00.533839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:00.534483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:00.535850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:51:00.547910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:51:00.549218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:00.550489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 645/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:51:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 13:51:00.555284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:00.657614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfb938-79bc-4468-9001-849b70beaf80 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:00.690830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:00.691493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:00.691778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:00.692084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:00.692792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:51:00.693097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:00.694473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147532e-ef03-4e69-9fda-e0158b5bb87c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:00.699848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 646/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jul 15 13:51:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:51:00.705702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:00.804173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d51be-008e-40e4-8ba2-a6bf4f9f8726 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:00.836726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:00.837381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:00.837660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:00.837934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:00.838632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:51:00.838932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:00.840308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7f57-2030-4b53-8d0c-9decc7213895 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:00.844682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 646/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jul 15 13:51:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:51:02.826507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2778dd-7bb9-4da9-906d-9e7651446fb8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:02.827066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2778dd-7bb9-4da9-906d-9e7651446fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:02.827066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2778dd-7bb9-4da9-906d-9e7651446fb8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:02.828491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 647/1293] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:51:02.833561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8dc52-6c76-4a86-947a-400e4058f877 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:02.833950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8dc52-6c76-4a86-947a-400e4058f877 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:02.834123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8dc52-6c76-4a86-947a-400e4058f877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:02.851250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3ce8dc52-6c76-4a86-947a-400e4058f877 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:51:02.852447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3ce8dc52-6c76-4a86-947a-400e4058f877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:51:02.880926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:02.920919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 647/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:51:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:51:02.945430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2778dd-7bb9-4da9-906d-9e7651446fb8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:02.980634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:03.055856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2778dd-7bb9-4da9-906d-9e7651446fb8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:03.091828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adfa79-0b32-4f45-927d-aa2711886cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:03.092439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adfa79-0b32-4f45-927d-aa2711886cac None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:03.092723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adfa79-0b32-4f45-927d-aa2711886cac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:03.092995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adfa79-0b32-4f45-927d-aa2711886cac None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:03.093731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adfa79-0b32-4f45-927d-aa2711886cac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:03.095103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adfa79-0b32-4f45-927d-aa2711886cac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:03.110205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 648/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jul 15 13:51:02 2022] POST /identity/v3/services => generated 246 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 13:51:05.139263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6152f8-c437-4f39-ac3b-a72674a28f9d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:05.139919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6152f8-c437-4f39-ac3b-a72674a28f9d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:05.139919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6152f8-c437-4f39-ac3b-a72674a28f9d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:05.141268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 648/1296] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:51:05.146846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:05.147295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:05.147402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:05.163755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:51:05.164974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:51:05.236643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 649/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:51:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:51:05.259592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6152f8-c437-4f39-ac3b-a72674a28f9d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:05.363215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6152f8-c437-4f39-ac3b-a72674a28f9d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:05.397268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:05.397974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:05.398249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:05.398518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:05.399196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:05.400591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:51:05.412276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:51:05.413538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:05.414585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 649/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:51:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:51:05.420173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:05.438715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:05.519100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda34b2-9199-425d-abeb-272c7b17e38e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:05.552624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:05.553282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:05.553562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:05.553836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:05.554535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:51:05.554838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:05.556198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:05.561327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 650/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jul 15 13:51:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:51:05.567288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:05.598387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:05.665511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e771a88-aa51-49cd-8596-a176c6e5975b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:05.697866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:05.698535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:05.698816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:05.699087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:05.699866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] RBAC: Adding query filter params (interface=public, service_id=1871f869e16849658c0c18799c767e0b, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:51:05.700169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:05.701553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ade001-cf1f-4977-b698-f5dbb42ff47a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:05.706519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 650/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jul 15 13:51:05 2022] GET /identity/v3/endpoints?service_id=1871f869e16849658c0c18799c767e0b&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:51:05.712196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:05.795025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:05.813993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73679c39-eb56-4cec-bad4-d303577236a8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:05.848490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2dffd-aa5b-48d4-9726-bb337d714129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:05.849409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2dffd-aa5b-48d4-9726-bb337d714129 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:05.849692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2dffd-aa5b-48d4-9726-bb337d714129 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:05.849964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2dffd-aa5b-48d4-9726-bb337d714129 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:05.850638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2dffd-aa5b-48d4-9726-bb337d714129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:05.852004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2dffd-aa5b-48d4-9726-bb337d714129 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:05.858876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 651/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jul 15 13:51:05 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 13:51:07.871722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad939c-e23c-4971-a9aa-b8bc2f983aec None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:07.872268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad939c-e23c-4971-a9aa-b8bc2f983aec None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:07.872268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad939c-e23c-4971-a9aa-b8bc2f983aec None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:07.873725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 651/1302] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:51:07.879742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:07.880341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:07.880341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:07.898000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:51:07.899199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:51:07.978497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 652/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:51:07 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 13:51:08.001391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad939c-e23c-4971-a9aa-b8bc2f983aec None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:08.096370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:08.104106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad939c-e23c-4971-a9aa-b8bc2f983aec None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:08.146045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:08.146762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:08.147040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:08.147311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:08.147970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:08.149384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 13:51:08.161836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 13:51:08.162893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 15 13:51:08.164212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 652/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jul 15 13:51:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 13:51:08.170000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:08.243786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:08.266310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec93bbd-f23b-40ea-930a-68864e8c9de8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:08.298623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:08.299269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:08.299548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:08.299822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:08.300539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:51:08.300842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:08.302190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98886063-7386-4400-b7c3-eb59e5567138 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:08.307156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 653/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jul 15 13:51:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:51:08.312912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:51:08.410612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b634d0-fc65-4029-ad6f-1592b8dbecd5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:51:08.442918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743184e-e847-4981-baa7-d5a8e1aa50d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:51:08.443567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743184e-e847-4981-baa7-d5a8e1aa50d7 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:51:08.443853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743184e-e847-4981-baa7-d5a8e1aa50d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:51:08.444133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743184e-e847-4981-baa7-d5a8e1aa50d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:51:08.444912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743184e-e847-4981-baa7-d5a8e1aa50d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:51:08.446288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743184e-e847-4981-baa7-d5a8e1aa50d7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:51:08.464246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 653/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jul 15 13:51:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:51:18.304407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=30 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:51:18.465624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:16.534355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b109d-2ebf-4736-b1db-aa9a21188f0f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:16.534928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b109d-2ebf-4736-b1db-aa9a21188f0f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:16.534928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b109d-2ebf-4736-b1db-aa9a21188f0f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:16.541335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 654/1307] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:16.542733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194b802-5cf7-4f95-bfc2-a405108e76e4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:16.543191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194b802-5cf7-4f95-bfc2-a405108e76e4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:16.543288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194b802-5cf7-4f95-bfc2-a405108e76e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:16.629732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d194b802-5cf7-4f95-bfc2-a405108e76e4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:16.631126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d194b802-5cf7-4f95-bfc2-a405108e76e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:16.765089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 654/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:52:16.807687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cc3b0-70af-4165-bdf9-40cfedbd13b5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:16.808114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cc3b0-70af-4165-bdf9-40cfedbd13b5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:16.808640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cc3b0-70af-4165-bdf9-40cfedbd13b5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:16.811067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 655/1309] 10.222.0.16 () {54 vars in 964 bytes} [Fri Jul 15 13:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:16.818313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22b7d-3592-41f6-a719-dc0673b927ec None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:16.818608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22b7d-3592-41f6-a719-dc0673b927ec None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:16.818891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc22b7d-3592-41f6-a719-dc0673b927ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:16.921059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0fc22b7d-3592-41f6-a719-dc0673b927ec None None] MFA Rules not processed for user `393c4bea16074727958ea40ab497948d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:16.922351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0fc22b7d-3592-41f6-a719-dc0673b927ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:16.947716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:17.024374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 655/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Fri Jul 15 13:52:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:17.034080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cc3b0-70af-4165-bdf9-40cfedbd13b5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:17.035817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3a5cc3b0-70af-4165-bdf9-40cfedbd13b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:17.169556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cc3b0-70af-4165-bdf9-40cfedbd13b5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:17.263207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:17.264040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:17.264487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:17.264712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:17.266576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:17.326185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:17.388425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:17.390028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8a7b-b587-4d24-9d74-8fa6fb206a71 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:17.410876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 656/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:20.053531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97de96-4be3-44f1-a9cb-791459f04aab None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:20.054115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97de96-4be3-44f1-a9cb-791459f04aab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:20.054115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97de96-4be3-44f1-a9cb-791459f04aab None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:20.055558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 656/1312] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:20.061413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0e93-361b-4457-b89c-93c80bba98a0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:20.061967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0e93-361b-4457-b89c-93c80bba98a0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:20.061967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d0e93-361b-4457-b89c-93c80bba98a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:20.146967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ae0d0e93-361b-4457-b89c-93c80bba98a0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:20.148183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ae0d0e93-361b-4457-b89c-93c80bba98a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:20.256626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 657/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:20.298121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97de96-4be3-44f1-a9cb-791459f04aab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:20.299841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cb97de96-4be3-44f1-a9cb-791459f04aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:20.421975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97de96-4be3-44f1-a9cb-791459f04aab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:20.516702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:20.520790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:20.521815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:20.522212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:20.523008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:20.525751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:20.656376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:20.657871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303b2b4-4eb4-48a2-a235-3141f8d8dcdd service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:20.678018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 657/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:23.108505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679ac42-7dc2-493c-9762-029fc4121194 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:23.109092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679ac42-7dc2-493c-9762-029fc4121194 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:23.109092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679ac42-7dc2-493c-9762-029fc4121194 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:23.110507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 658/1315] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:52:23.115893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74868bfb-3f08-45f6-86f6-dd71d41ba445 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:23.116376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74868bfb-3f08-45f6-86f6-dd71d41ba445 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:23.116500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74868bfb-3f08-45f6-86f6-dd71d41ba445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:23.188340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-74868bfb-3f08-45f6-86f6-dd71d41ba445 None None] MFA Rules not processed for user `1c3a6fffeecf41fb83087b2815dfe4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:23.189570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-74868bfb-3f08-45f6-86f6-dd71d41ba445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:23.248739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:23.294169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 658/1316] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:23.334205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679ac42-7dc2-493c-9762-029fc4121194 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:23.335759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c679ac42-7dc2-493c-9762-029fc4121194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:23.346355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:23.450305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679ac42-7dc2-493c-9762-029fc4121194 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:23.526779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:23.527348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:23.527632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:23.527911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:23.529585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:23.592542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:23.639342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:23.640806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25fdebd-cd2b-4327-966c-9ac93be32d6c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:23.659149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 659/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:26.296736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e01c04-0de0-4b19-a02d-457fdb42faa7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:26.297331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e01c04-0de0-4b19-a02d-457fdb42faa7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:26.297331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e01c04-0de0-4b19-a02d-457fdb42faa7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:26.298865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 659/1318] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:26.304781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1dacc-3c75-4c43-9d1e-bc1673b47d9d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:26.305406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1dacc-3c75-4c43-9d1e-bc1673b47d9d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:26.305406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1dacc-3c75-4c43-9d1e-bc1673b47d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:26.382039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1ea1dacc-3c75-4c43-9d1e-bc1673b47d9d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:26.383262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1ea1dacc-3c75-4c43-9d1e-bc1673b47d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:26.481834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 660/1319] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:26.520529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e01c04-0de0-4b19-a02d-457fdb42faa7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:26.522076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c6e01c04-0de0-4b19-a02d-457fdb42faa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:26.632048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e01c04-0de0-4b19-a02d-457fdb42faa7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:26.710352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:26.711012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:26.711294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:26.711575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:26.713855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:26.721997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:26.830677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:26.832174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c4c98-b3b3-4352-a95b-cbfbfe7259ba service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:26.851822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 660/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:29.626997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ed47d-0be7-4aaf-b659-37b7602c7614 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:29.627599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ed47d-0be7-4aaf-b659-37b7602c7614 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:29.627599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ed47d-0be7-4aaf-b659-37b7602c7614 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:29.629135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 661/1321] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:29.634574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf88a9-e9bb-4e07-84fb-2a7e022a2a5c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:29.635090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf88a9-e9bb-4e07-84fb-2a7e022a2a5c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:29.635195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf88a9-e9bb-4e07-84fb-2a7e022a2a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:29.713861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3ccf88a9-e9bb-4e07-84fb-2a7e022a2a5c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:29.715105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3ccf88a9-e9bb-4e07-84fb-2a7e022a2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:29.781655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:29.830163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 661/1322] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:29.867153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ed47d-0be7-4aaf-b659-37b7602c7614 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:29.868927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-684ed47d-0be7-4aaf-b659-37b7602c7614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:29.889485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:29.982270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ed47d-0be7-4aaf-b659-37b7602c7614 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:30.060192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:30.060882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:30.061163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:30.061436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:30.063144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:30.142760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:30.190629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:30.192096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19655acd-b93d-4521-a25c-0c96c33c6bb8 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:30.212945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 662/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:32.316408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79183c-d646-43a9-b2ea-a37558497a50 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:32.316948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79183c-d646-43a9-b2ea-a37558497a50 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:32.316948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79183c-d646-43a9-b2ea-a37558497a50 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:32.318405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 662/1324] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:32.324449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5274-6ad4-47fa-b108-6990cbdbaab7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:32.324895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5274-6ad4-47fa-b108-6990cbdbaab7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:32.324999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5274-6ad4-47fa-b108-6990cbdbaab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:32.396088 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-076f5274-6ad4-47fa-b108-6990cbdbaab7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:32.397329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-076f5274-6ad4-47fa-b108-6990cbdbaab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:32.495050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 663/1325] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:32.531149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79183c-d646-43a9-b2ea-a37558497a50 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:32.532833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1a79183c-d646-43a9-b2ea-a37558497a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:32.640749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79183c-d646-43a9-b2ea-a37558497a50 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:32.717343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:32.718003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:32.718286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:32.718557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:32.720320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:32.739288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:32.834322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:32.835756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b43ec8-315f-409a-b3a1-9a73a58b75eb service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:32.859604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 663/1326] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:34.927253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09eaf-e6ff-4bd6-bba8-17f8e438bf4a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:34.927761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09eaf-e6ff-4bd6-bba8-17f8e438bf4a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:34.927761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09eaf-e6ff-4bd6-bba8-17f8e438bf4a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:34.929314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 664/1327] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:34.934460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba707a8-bcc4-417f-99e0-af2169573355 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:34.934876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba707a8-bcc4-417f-99e0-af2169573355 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:34.935008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba707a8-bcc4-417f-99e0-af2169573355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:35.002433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6ba707a8-bcc4-417f-99e0-af2169573355 None None] MFA Rules not processed for user `1c3a6fffeecf41fb83087b2815dfe4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:35.004564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ba707a8-bcc4-417f-99e0-af2169573355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:35.078427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:35.112668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 664/1328] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:35.151980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09eaf-e6ff-4bd6-bba8-17f8e438bf4a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:35.153674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d8e09eaf-e6ff-4bd6-bba8-17f8e438bf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:35.187019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:35.267854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09eaf-e6ff-4bd6-bba8-17f8e438bf4a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:35.394624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:35.395286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:35.395573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:35.395858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:35.397562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:35.477311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:35.511590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:35.513081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785195a-f02c-4ab6-b279-918f1a125143 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:35.531773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 665/1329] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:37.835115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ea386-1aab-4c06-81c3-aa3cd2eab95d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:37.835675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ea386-1aab-4c06-81c3-aa3cd2eab95d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:37.835675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ea386-1aab-4c06-81c3-aa3cd2eab95d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:37.837093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 665/1330] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:37.842982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00547f-bd0e-4758-839f-ea6ecb31fc4a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:37.843420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00547f-bd0e-4758-839f-ea6ecb31fc4a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:37.843532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00547f-bd0e-4758-839f-ea6ecb31fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:37.910834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6f00547f-bd0e-4758-839f-ea6ecb31fc4a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:37.912041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6f00547f-bd0e-4758-839f-ea6ecb31fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:38.009090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 666/1331] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:38.045581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ea386-1aab-4c06-81c3-aa3cd2eab95d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:38.047120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fa6ea386-1aab-4c06-81c3-aa3cd2eab95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:38.151893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ea386-1aab-4c06-81c3-aa3cd2eab95d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:38.224762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:38.225677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:38.225957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:38.226229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:38.227964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:38.260600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:38.348553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:38.350246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff1e13-486c-46a3-8ab9-871d5e8de59a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:38.368250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 666/1332] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:40.420523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89150aa-5d7b-4043-9b02-5a631a0df718 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:40.421030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89150aa-5d7b-4043-9b02-5a631a0df718 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:40.421030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89150aa-5d7b-4043-9b02-5a631a0df718 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:40.422394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 667/1333] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:40.427720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727b88-4096-4a79-be76-5b9445142726 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:40.428153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727b88-4096-4a79-be76-5b9445142726 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:40.428278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727b88-4096-4a79-be76-5b9445142726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:40.501679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0d727b88-4096-4a79-be76-5b9445142726 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:40.502896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0d727b88-4096-4a79-be76-5b9445142726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:40.578702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:40.602783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 667/1334] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:40.656491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515763f-96df-4373-8cf7-fe2877bb9df5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:40.656877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515763f-96df-4373-8cf7-fe2877bb9df5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:40.657040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515763f-96df-4373-8cf7-fe2877bb9df5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:40.658405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 668/1335] 10.222.0.16 () {54 vars in 964 bytes} [Fri Jul 15 13:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:40.664494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3e5a-f85f-4bf8-bb5d-d4a6e8b55c1d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:40.664783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3e5a-f85f-4bf8-bb5d-d4a6e8b55c1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:40.665060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3e5a-f85f-4bf8-bb5d-d4a6e8b55c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:40.740154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d48f3e5a-f85f-4bf8-bb5d-d4a6e8b55c1d None None] MFA Rules not processed for user `393c4bea16074727958ea40ab497948d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:40.741314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d48f3e5a-f85f-4bf8-bb5d-d4a6e8b55c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:40.815918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 668/1336] 10.222.0.16 () {58 vars in 1061 bytes} [Fri Jul 15 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:40.825142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515763f-96df-4373-8cf7-fe2877bb9df5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:40.826703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e515763f-96df-4373-8cf7-fe2877bb9df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:40.871730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:40.950298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515763f-96df-4373-8cf7-fe2877bb9df5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:41.026333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:41.026894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:41.027175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:41.027452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:41.029137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:41.125049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:41.155228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:41.156718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43b1e1-f084-41f3-b334-ae6189e96d98 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:41.175670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 669/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:44.061179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f12b9-740e-46ea-8ab8-030a77d3a363 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:44.061763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f12b9-740e-46ea-8ab8-030a77d3a363 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:44.061763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f12b9-740e-46ea-8ab8-030a77d3a363 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:44.063129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 669/1338] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:44.068322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681543d-81ef-4b16-b089-6e34dea27a88 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:44.068770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681543d-81ef-4b16-b089-6e34dea27a88 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:44.068902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681543d-81ef-4b16-b089-6e34dea27a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:44.134869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b681543d-81ef-4b16-b089-6e34dea27a88 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:44.136082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b681543d-81ef-4b16-b089-6e34dea27a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:44.242028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 670/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:44.278501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f12b9-740e-46ea-8ab8-030a77d3a363 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:44.280046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-034f12b9-740e-46ea-8ab8-030a77d3a363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:44.355764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:44.383989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f12b9-740e-46ea-8ab8-030a77d3a363 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:44.459403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:44.460049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:44.460331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:44.460635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:44.462298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:44.588682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:44.590126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb1c43-9449-4c57-9840-1532b5467c7d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:44.610648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 670/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:48.607216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf352e8e-6160-4909-9caf-96f912f37162 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:48.607747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf352e8e-6160-4909-9caf-96f912f37162 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:48.607747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf352e8e-6160-4909-9caf-96f912f37162 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:48.609144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 671/1341] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:48.615075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240627-7ea8-4c26-b7b5-5d64720bb923 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:48.615508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240627-7ea8-4c26-b7b5-5d64720bb923 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:48.615816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60240627-7ea8-4c26-b7b5-5d64720bb923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:48.630743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:48.709988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-60240627-7ea8-4c26-b7b5-5d64720bb923 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:48.711221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60240627-7ea8-4c26-b7b5-5d64720bb923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:48.819938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 671/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:48.860332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf352e8e-6160-4909-9caf-96f912f37162 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:48.861990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cf352e8e-6160-4909-9caf-96f912f37162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:48.916126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:48.980181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf352e8e-6160-4909-9caf-96f912f37162 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:49.062195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:49.062864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:49.063138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:49.063404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:49.065109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:49.177027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:49.197774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:49.199224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792c706-9af9-4326-9ec6-b09be2e26a3b service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:49.222072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 672/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:52.083411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584b113-b86f-44b6-8057-3a78e8e61555 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:52.083968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584b113-b86f-44b6-8057-3a78e8e61555 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:52.084048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584b113-b86f-44b6-8057-3a78e8e61555 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:52.085538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 672/1344] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:52.092471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9aa0c-9890-4c99-95d6-d9e25b71aee9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:52.093061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9aa0c-9890-4c99-95d6-d9e25b71aee9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:52.093159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9aa0c-9890-4c99-95d6-d9e25b71aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:52.166730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-adc9aa0c-9890-4c99-95d6-d9e25b71aee9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:52.167950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-adc9aa0c-9890-4c99-95d6-d9e25b71aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:52.277584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 673/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:52:52.314923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584b113-b86f-44b6-8057-3a78e8e61555 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:52.316445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b584b113-b86f-44b6-8057-3a78e8e61555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:52.410882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:52.429774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584b113-b86f-44b6-8057-3a78e8e61555 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:52.514757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:52.515413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:52.515695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:52.516091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:52.517801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:52.635458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:52.637700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49747-c6cb-4897-826f-68da9aa67651 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:52.658410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 673/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:52:57.136486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b17f6-f8ce-44f1-9c22-ab051dadb939 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:57.137246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b17f6-f8ce-44f1-9c22-ab051dadb939 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:57.137510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b17f6-f8ce-44f1-9c22-ab051dadb939 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:57.143877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 674/1347] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:52:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:52:57.149360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8537c8-ece7-49f7-8677-7171eef42e0c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:57.150019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8537c8-ece7-49f7-8677-7171eef42e0c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:57.150019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8537c8-ece7-49f7-8677-7171eef42e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:57.166447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:57.235622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c8537c8-ece7-49f7-8677-7171eef42e0c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:52:57.236924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c8537c8-ece7-49f7-8677-7171eef42e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:57.377086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 674/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:52:57.417195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b17f6-f8ce-44f1-9c22-ab051dadb939 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:52:57.418861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8f2b17f6-f8ce-44f1-9c22-ab051dadb939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:57.499282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:57.570330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b17f6-f8ce-44f1-9c22-ab051dadb939 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:52:57.677109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:52:57.678158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:52:57.678660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:52:57.679139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:52:57.681281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:52:57.826986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:52:57.832915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:52:57.835038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a79ac-adf3-47ec-9ebc-083d18142730 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:52:57.865257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 675/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:03.218964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d76d7-216d-4510-a6d0-e1b1b0600ccf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:03.219791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d76d7-216d-4510-a6d0-e1b1b0600ccf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:03.220123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d76d7-216d-4510-a6d0-e1b1b0600ccf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:03.224166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 675/1350] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 13:53:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:03.235071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110da3-a844-40e3-9513-311b30bb12b1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:03.236976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110da3-a844-40e3-9513-311b30bb12b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:03.237611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b110da3-a844-40e3-9513-311b30bb12b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:03.348921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5b110da3-a844-40e3-9513-311b30bb12b1 None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:03.350293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5b110da3-a844-40e3-9513-311b30bb12b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:03.569524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 676/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 13:53:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:07.374914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=59 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:09.234255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317cc0-2668-4025-aaa8-20a7b4eb2160 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:09.234871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317cc0-2668-4025-aaa8-20a7b4eb2160 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:09.234871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317cc0-2668-4025-aaa8-20a7b4eb2160 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:09.236515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 676/1352] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 13:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:09.241686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186528a8-aeea-4942-b996-6734329345f4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:09.242188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186528a8-aeea-4942-b996-6734329345f4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:09.242275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186528a8-aeea-4942-b996-6734329345f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:09.301558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:09.327997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-186528a8-aeea-4942-b996-6734329345f4 None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:09.329516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-186528a8-aeea-4942-b996-6734329345f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:09.447667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 677/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 13:53:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:53:10.486337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d568-68cc-4a18-aa11-083f7dd55d49 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:10.486884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d568-68cc-4a18-aa11-083f7dd55d49 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:10.486884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d568-68cc-4a18-aa11-083f7dd55d49 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:10.488287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 677/1354] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 13:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:10.494315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad51a-b9b2-4aef-bc66-a7b83fb1ec17 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:10.494779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad51a-b9b2-4aef-bc66-a7b83fb1ec17 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:10.494899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cad51a-b9b2-4aef-bc66-a7b83fb1ec17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:10.570228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b7cad51a-b9b2-4aef-bc66-a7b83fb1ec17 None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:10.571577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b7cad51a-b9b2-4aef-bc66-a7b83fb1ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:10.615924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:10.683469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 678/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 13:53:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:14.150256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879f65-8a3d-4328-892f-6df8b493e064 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:14.151127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879f65-8a3d-4328-892f-6df8b493e064 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:14.151412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f879f65-8a3d-4328-892f-6df8b493e064 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:14.154385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 678/1356] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:53:14.166055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1023-f7f4-4cef-93b5-8276ef370e03 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:14.167690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1023-f7f4-4cef-93b5-8276ef370e03 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:14.169412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a1023-f7f4-4cef-93b5-8276ef370e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:14.291789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e71a1023-f7f4-4cef-93b5-8276ef370e03 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:14.293162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e71a1023-f7f4-4cef-93b5-8276ef370e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:14.420514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 679/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:14.473133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07561b-9262-4965-8c21-b3631143b6c4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:14.473332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07561b-9262-4965-8c21-b3631143b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:14.473611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07561b-9262-4965-8c21-b3631143b6c4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:14.475036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 679/1358] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jul 15 13:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:14.486076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac607-4fb7-44f4-9437-ecb345aa50a5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:14.486375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac607-4fb7-44f4-9437-ecb345aa50a5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:14.486642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac607-4fb7-44f4-9437-ecb345aa50a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:14.511897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:14.584151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8bdac607-4fb7-44f4-9437-ecb345aa50a5 None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:14.585701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8bdac607-4fb7-44f4-9437-ecb345aa50a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:14.719994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 680/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jul 15 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:14.728255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07561b-9262-4965-8c21-b3631143b6c4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:14.729939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dd07561b-9262-4965-8c21-b3631143b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:14.891106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07561b-9262-4965-8c21-b3631143b6c4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:14.984431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:15.053427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:15.054240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:15.054591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:15.054943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:15.056842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:15.169673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:15.170687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbed15-e283-4286-9865-adaa9e299a8e service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:15.191494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 680/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:17.554074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3db67-62da-4b9a-a2d4-588744a0f096 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:17.554718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3db67-62da-4b9a-a2d4-588744a0f096 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:17.554718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3db67-62da-4b9a-a2d4-588744a0f096 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:17.556200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 681/1361] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:17.561648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5b6a-46d0-485f-a06a-ed63e51b6428 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:17.562089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5b6a-46d0-485f-a06a-ed63e51b6428 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:17.562190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5b6a-46d0-485f-a06a-ed63e51b6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:17.637952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-234a5b6a-46d0-485f-a06a-ed63e51b6428 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:17.638987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-234a5b6a-46d0-485f-a06a-ed63e51b6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:17.684082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:17.770781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 681/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:53:17.819084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3db67-62da-4b9a-a2d4-588744a0f096 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:17.823931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7dd3db67-62da-4b9a-a2d4-588744a0f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:17.943886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:18.056190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3db67-62da-4b9a-a2d4-588744a0f096 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:18.208567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:18.209072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:18.209359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:18.209641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:18.211376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:18.242350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:18.353473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:18.356597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71170170-b4fb-40c0-b120-c7d686baf004 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:18.394478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 682/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:20.969564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279af1f-83cc-4852-a929-f382514be253 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:20.970393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279af1f-83cc-4852-a929-f382514be253 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:20.970393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279af1f-83cc-4852-a929-f382514be253 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:20.972751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 682/1364] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:20.982798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daba094-76db-4532-95d2-8867bea2fbc1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:20.983281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daba094-76db-4532-95d2-8867bea2fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:20.983482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daba094-76db-4532-95d2-8867bea2fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:21.108900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2daba094-76db-4532-95d2-8867bea2fbc1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:21.110887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2daba094-76db-4532-95d2-8867bea2fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:21.293099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:21.304857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 683/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:21.365269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d6b0a-d287-4a88-aac5-2a1562a4af68 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:21.365547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d6b0a-d287-4a88-aac5-2a1562a4af68 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:21.366654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d6b0a-d287-4a88-aac5-2a1562a4af68 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:21.368057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 683/1366] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jul 15 13:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:21.382724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39877c30-bbc1-4ad2-aab4-72497360db97 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:21.383092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39877c30-bbc1-4ad2-aab4-72497360db97 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:21.383334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39877c30-bbc1-4ad2-aab4-72497360db97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:21.513882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-39877c30-bbc1-4ad2-aab4-72497360db97 None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:21.515821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-39877c30-bbc1-4ad2-aab4-72497360db97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:21.680566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 684/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jul 15 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:21.685532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d6b0a-d287-4a88-aac5-2a1562a4af68 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:21.687103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bb3d6b0a-d287-4a88-aac5-2a1562a4af68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:21.925102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d6b0a-d287-4a88-aac5-2a1562a4af68 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:21.968084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:22.069963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ad575-7207-497b-812d-b613abb8701b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:22.071956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ad575-7207-497b-812d-b613abb8701b service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:22.072469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ad575-7207-497b-812d-b613abb8701b service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:22.072941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ad575-7207-497b-812d-b613abb8701b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:22.074649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-765ad575-7207-497b-812d-b613abb8701b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:22.171989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ad575-7207-497b-812d-b613abb8701b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:22.173678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ad575-7207-497b-812d-b613abb8701b service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:22.197255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 684/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:22.723513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4168-30b0-49fd-9b8e-1e39f1959674 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:22.724037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4168-30b0-49fd-9b8e-1e39f1959674 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:22.724121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4168-30b0-49fd-9b8e-1e39f1959674 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:22.725932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 685/1369] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:22.731860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a73530-218e-43e2-9117-ba38a4164a4f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:22.732169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a73530-218e-43e2-9117-ba38a4164a4f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:22.732468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a73530-218e-43e2-9117-ba38a4164a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:22.769987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:22.816036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-17a73530-218e-43e2-9117-ba38a4164a4f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:22.817309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-17a73530-218e-43e2-9117-ba38a4164a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:22.936541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 685/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:23.266670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728e42d-5a83-4433-8161-c070778811b3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:23.266979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728e42d-5a83-4433-8161-c070778811b3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:23.267401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728e42d-5a83-4433-8161-c070778811b3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:23.269033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 686/1371] 10.222.0.16 () {54 vars in 963 bytes} [Fri Jul 15 13:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:23.274494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bb558-0367-4965-b5f3-3d95ca63b657 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:23.275009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bb558-0367-4965-b5f3-3d95ca63b657 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:23.275106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bb558-0367-4965-b5f3-3d95ca63b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:23.365878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f91bb558-0367-4965-b5f3-3d95ca63b657 None None] MFA Rules not processed for user `f0e94eecf28f42468e14bbd0c4eb3777`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:23.367118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f91bb558-0367-4965-b5f3-3d95ca63b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:23.421705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:23.462404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 686/1372] 10.222.0.16 () {58 vars in 1060 bytes} [Fri Jul 15 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:23.470231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728e42d-5a83-4433-8161-c070778811b3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:23.471896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a728e42d-5a83-4433-8161-c070778811b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:23.528850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:23.590917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728e42d-5a83-4433-8161-c070778811b3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:23.678335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:23.678913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:23.679201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:23.679484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:23.681298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:23.817265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:23.846346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:23.849065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d55ec-71ce-4990-80f3-4e78fa8291ce service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:23.898336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 687/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:24.318950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ddb4e-ad20-4322-a58a-9631bd328458 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:24.319541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ddb4e-ad20-4322-a58a-9631bd328458 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:24.319643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ddb4e-ad20-4322-a58a-9631bd328458 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:24.321559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 687/1374] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:24.328609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e21e-88de-4f8e-9263-cd8ed15e2cb8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:24.328900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e21e-88de-4f8e-9263-cd8ed15e2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:24.329178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e21e-88de-4f8e-9263-cd8ed15e2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:24.411471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-18c8e21e-88de-4f8e-9263-cd8ed15e2cb8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:24.412748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-18c8e21e-88de-4f8e-9263-cd8ed15e2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:24.522093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 688/1375] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:24.569170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ddb4e-ad20-4322-a58a-9631bd328458 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:24.570625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-183ddb4e-ad20-4322-a58a-9631bd328458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:24.782740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ddb4e-ad20-4322-a58a-9631bd328458 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:24.851159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:24.927630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:24.928302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:24.928645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:24.928919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:24.931380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:25.033514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:25.038744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f4ee3-2501-46c4-8ded-8f284fa3aecf service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:25.066850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 688/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:27.163802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752df342-1d3d-446d-9720-6b87abb866b6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:27.164355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752df342-1d3d-446d-9720-6b87abb866b6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:27.164355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752df342-1d3d-446d-9720-6b87abb866b6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:27.166257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 689/1377] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:27.171938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02091b-8771-4eb8-98fc-471247555851 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:27.172439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02091b-8771-4eb8-98fc-471247555851 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:27.172552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02091b-8771-4eb8-98fc-471247555851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:27.245973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5c02091b-8771-4eb8-98fc-471247555851 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:27.247183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5c02091b-8771-4eb8-98fc-471247555851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:27.254277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:27.351214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 689/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:27.396059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752df342-1d3d-446d-9720-6b87abb866b6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:27.397631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-752df342-1d3d-446d-9720-6b87abb866b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:27.478734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:27.612825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752df342-1d3d-446d-9720-6b87abb866b6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:27.757264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:27.758007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:27.759109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:27.759495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:27.761315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:27.784464 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:27.864547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:27.866167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291f785-ff4f-4303-bf47-02608fd74176 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:27.886652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 690/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:29.984661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c3786-95ef-428d-9ef4-d98b761a127b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:29.985333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c3786-95ef-428d-9ef4-d98b761a127b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:29.985333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c3786-95ef-428d-9ef4-d98b761a127b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:29.986592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 690/1380] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:29.992019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4647eb-c17b-4a40-b057-2ba4409aaf07 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:29.992629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4647eb-c17b-4a40-b057-2ba4409aaf07 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:29.992629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4647eb-c17b-4a40-b057-2ba4409aaf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:30.066083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8d4647eb-c17b-4a40-b057-2ba4409aaf07 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:30.067403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8d4647eb-c17b-4a40-b057-2ba4409aaf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:30.186075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:30.194145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 691/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:30.241580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c3786-95ef-428d-9ef4-d98b761a127b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:30.243130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a34c3786-95ef-428d-9ef4-d98b761a127b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:30.411367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:30.424883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c3786-95ef-428d-9ef4-d98b761a127b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:30.597503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:30.598168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:30.598449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:30.598720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:30.600564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:30.706768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:30.708295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a29262-f0f4-41f0-836f-dcedba6e66b6 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:30.720331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:30.735677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 691/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:32.842663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50bc33-b626-4cca-bfea-e8e4eb56e633 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:32.843524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50bc33-b626-4cca-bfea-e8e4eb56e633 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:32.843524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50bc33-b626-4cca-bfea-e8e4eb56e633 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:32.844662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 692/1383] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:32.850200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4374a-0d93-4b26-b3f6-77047a6c47fd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:32.851025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4374a-0d93-4b26-b3f6-77047a6c47fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:32.851025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4374a-0d93-4b26-b3f6-77047a6c47fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:32.936982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-40d4374a-0d93-4b26-b3f6-77047a6c47fd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:32.938808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-40d4374a-0d93-4b26-b3f6-77047a6c47fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:33.066217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 692/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:33.113366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50bc33-b626-4cca-bfea-e8e4eb56e633 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:33.114947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-de50bc33-b626-4cca-bfea-e8e4eb56e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:33.329036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50bc33-b626-4cca-bfea-e8e4eb56e633 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:33.451831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:33.529837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:33.530707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:33.531174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:33.531582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:33.533738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:33.661140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:33.663446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f2017-0df4-429e-a744-5b77a891bdc0 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:33.697541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 693/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:35.809459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de220a-fe7f-4091-9c7e-bcfb0cafb8df None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:35.810357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de220a-fe7f-4091-9c7e-bcfb0cafb8df None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:35.810357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de220a-fe7f-4091-9c7e-bcfb0cafb8df None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:35.812074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 693/1386] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:35.818352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7269d-06c8-46b9-b49d-0274f50380eb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:35.818352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7269d-06c8-46b9-b49d-0274f50380eb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:35.819136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7269d-06c8-46b9-b49d-0274f50380eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:35.907983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b4a7269d-06c8-46b9-b49d-0274f50380eb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:35.910158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b4a7269d-06c8-46b9-b49d-0274f50380eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:35.952382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:36.030311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 694/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:36.095168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de220a-fe7f-4091-9c7e-bcfb0cafb8df None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:36.097555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-88de220a-fe7f-4091-9c7e-bcfb0cafb8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:36.237220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:36.364537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de220a-fe7f-4091-9c7e-bcfb0cafb8df None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:36.524874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:36.525595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:36.525892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:36.526184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:36.527993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:36.573757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:36.640449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:36.642066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc81d-c2a8-405e-a970-df7e10ed9e61 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:36.669364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 694/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:38.791254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0e38-fb4f-4074-9080-22a4177f177a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:38.791814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0e38-fb4f-4074-9080-22a4177f177a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:38.791814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0e38-fb4f-4074-9080-22a4177f177a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:38.793252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 695/1389] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:38.798982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3189e8-62a2-4bc8-97bd-3085a5cebd5f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:38.799662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3189e8-62a2-4bc8-97bd-3085a5cebd5f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:38.799662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3189e8-62a2-4bc8-97bd-3085a5cebd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:38.876819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5f3189e8-62a2-4bc8-97bd-3085a5cebd5f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:38.878056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5f3189e8-62a2-4bc8-97bd-3085a5cebd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:39.031333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 695/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:39.079045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a0e38-fb4f-4074-9080-22a4177f177a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:39.080700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c16a0e38-fb4f-4074-9080-22a4177f177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:39.299311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd254d4-6bed-4ff8-b0ad-4b8a1e5a33a1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:39.299702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd254d4-6bed-4ff8-b0ad-4b8a1e5a33a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:39.300102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd254d4-6bed-4ff8-b0ad-4b8a1e5a33a1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:39.302124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 696/1391] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:53:39.308290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9ff60-3c96-4c51-b537-c655806ea17d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:39.308857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9ff60-3c96-4c51-b537-c655806ea17d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:39.309140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9ff60-3c96-4c51-b537-c655806ea17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:39.325593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:39.330886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a0e38-fb4f-4074-9080-22a4177f177a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:39.351327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:39.391584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-59d9ff60-3c96-4c51-b537-c655806ea17d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:39.392903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59d9ff60-3c96-4c51-b537-c655806ea17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:39.467481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:39.468554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:39.469042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:39.469519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:39.472630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:39.507404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 697/1392] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:53:39.566357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac365-278c-4335-9a85-818d75b4d2b5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:39.566639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac365-278c-4335-9a85-818d75b4d2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:39.566919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ac365-278c-4335-9a85-818d75b4d2b5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:39.568304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 698/1393] 10.222.0.16 () {54 vars in 960 bytes} [Fri Jul 15 13:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:53:39.573412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc40461-380a-4ab0-8a22-1a36da6bf8fb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:39.573696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc40461-380a-4ab0-8a22-1a36da6bf8fb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:39.573971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc40461-380a-4ab0-8a22-1a36da6bf8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:39.602454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:39.604315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:39.623660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 696/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:39.663577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cfc40461-380a-4ab0-8a22-1a36da6bf8fb None None] MFA Rules not processed for user `deb87accdcab4759add4b76958d151a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:39.665347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cfc40461-380a-4ab0-8a22-1a36da6bf8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:39.715811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:39.774329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 699/1395] 10.222.0.16 () {58 vars in 1057 bytes} [Fri Jul 15 13:53:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:53:39.782406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:39.783745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:39.810390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:39.875182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f533d-6274-4d9c-ba06-f8b71ce06724 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:39.952510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:39.953167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:39.953451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:39.953732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:39.955412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:40.056915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:40.069960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:40.071323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2b5c7-c476-4f45-b747-d9ee4adde971 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:40.090597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 697/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:40.809110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b524c-ad01-4e7f-872f-2fb514347945 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:40.809761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b524c-ad01-4e7f-872f-2fb514347945 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:40.810176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b524c-ad01-4e7f-872f-2fb514347945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:40.881732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fb5b524c-ad01-4e7f-872f-2fb514347945 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:40.882964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fb5b524c-ad01-4e7f-872f-2fb514347945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:40.981261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 700/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:41.025213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431e12-4bb6-4609-9af7-7af7ac277160 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:41.025717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431e12-4bb6-4609-9af7-7af7ac277160 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:41.026142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf431e12-4bb6-4609-9af7-7af7ac277160 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:41.027604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 698/1398] 10.222.0.16 () {54 vars in 942 bytes} [Fri Jul 15 13:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:41.034545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9f49a-a665-441a-8356-dc8f72b7a1e3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:41.034841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9f49a-a665-441a-8356-dc8f72b7a1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:41.035122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9f49a-a665-441a-8356-dc8f72b7a1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:41.094736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:41.115250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-97e9f49a-a665-441a-8356-dc8f72b7a1e3 None None] MFA Rules not processed for user `f3eacd1356f448cca079f57f683f37bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:41.116482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97e9f49a-a665-441a-8356-dc8f72b7a1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:41.229935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 701/1399] 10.222.0.16 () {58 vars in 1039 bytes} [Fri Jul 15 13:53:41 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:41.239026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf431e12-4bb6-4609-9af7-7af7ac277160 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:41.240614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bf431e12-4bb6-4609-9af7-7af7ac277160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:41.397580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf431e12-4bb6-4609-9af7-7af7ac277160 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:41.492913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:41.561425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893123b1-234d-4545-a156-579859984c7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:41.562480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893123b1-234d-4545-a156-579859984c7b service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:41.562896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893123b1-234d-4545-a156-579859984c7b service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:41.563277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893123b1-234d-4545-a156-579859984c7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:41.565868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-893123b1-234d-4545-a156-579859984c7b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:41.679573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893123b1-234d-4545-a156-579859984c7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:41.681766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893123b1-234d-4545-a156-579859984c7b service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:41.691078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 699/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 13:53:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 13:53:41.747043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826283-e67d-4941-99a9-940046d6b113 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:41.747827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826283-e67d-4941-99a9-940046d6b113 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:41.748298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826283-e67d-4941-99a9-940046d6b113 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:41.751101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 702/1401] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:41.757086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591097-3d44-4ab8-804b-f024f2d81e55 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:41.757478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591097-3d44-4ab8-804b-f024f2d81e55 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:41.757770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591097-3d44-4ab8-804b-f024f2d81e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:41.873097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3f591097-3d44-4ab8-804b-f024f2d81e55 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:41.874380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3f591097-3d44-4ab8-804b-f024f2d81e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:41.965521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:42.021075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 700/1402] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:42.066623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826283-e67d-4941-99a9-940046d6b113 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:42.068184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e826283-e67d-4941-99a9-940046d6b113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:42.216712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:42.278485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826283-e67d-4941-99a9-940046d6b113 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:42.427197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:42.428218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:42.428534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:42.428822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:42.431270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:42.529702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:42.562927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:42.565569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec47c5-3dff-40c3-828c-a50abf156c69 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:42.590622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 703/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:44.753079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785e44c-e7ef-44bb-9c1d-dc779ed441cb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:44.753079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785e44c-e7ef-44bb-9c1d-dc779ed441cb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:44.753079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785e44c-e7ef-44bb-9c1d-dc779ed441cb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:44.755326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf53de-9b8b-40b2-9831-942348a53cc1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:44.755971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf53de-9b8b-40b2-9831-942348a53cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:44.755971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf53de-9b8b-40b2-9831-942348a53cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:44.759845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 701/1404] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:44 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:44.833735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-abbf53de-9b8b-40b2-9831-942348a53cc1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:44.835504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-abbf53de-9b8b-40b2-9831-942348a53cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:44.969832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 704/1405] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:45.019514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785e44c-e7ef-44bb-9c1d-dc779ed441cb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:45.021119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6785e44c-e7ef-44bb-9c1d-dc779ed441cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:45.216246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785e44c-e7ef-44bb-9c1d-dc779ed441cb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:45.261078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:45.357818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:45.358354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:45.358634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:45.358913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:45.360814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:45.467286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:45.469269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b04e83-906c-449f-b066-79369ea7720e service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:45.493762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 702/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:47.628354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31b9b-d9bb-492c-9807-835256ef2a35 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:47.628978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31b9b-d9bb-492c-9807-835256ef2a35 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:47.628978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31b9b-d9bb-492c-9807-835256ef2a35 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:47.630609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 705/1407] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:53:47.636196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d45e9-ff28-46e2-bb51-cb89cbc3bfe6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:47.636847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d45e9-ff28-46e2-bb51-cb89cbc3bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:47.636847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d45e9-ff28-46e2-bb51-cb89cbc3bfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:47.712119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:47.732585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-194d45e9-ff28-46e2-bb51-cb89cbc3bfe6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:53:47.733524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-194d45e9-ff28-46e2-bb51-cb89cbc3bfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:47.964828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 703/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:53:48.012052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd31b9b-d9bb-492c-9807-835256ef2a35 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:53:48.014285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9fd31b9b-d9bb-492c-9807-835256ef2a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:48.085116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:48.263401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd31b9b-d9bb-492c-9807-835256ef2a35 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:53:48.535644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:53:48.536216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:53:48.536548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:53:48.536830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:53:48.538497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:53:48.548231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:48.646735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:53:48.648935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd1f56-c020-42e6-8e51-a7e5136d3b9c service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:53:48.673283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 706/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:53:57.958429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=46 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:53:58.671044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=41 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:04.690980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d933532-85d9-46a3-a9f6-555710087d9a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:04.691533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d933532-85d9-46a3-a9f6-555710087d9a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:04.691533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d933532-85d9-46a3-a9f6-555710087d9a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:04.692981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 704/1410] 10.222.0.16 () {54 vars in 918 bytes} [Fri Jul 15 13:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:04.699039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919aa35-c39f-4c0b-8576-cbd9882dd21d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:04.699526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919aa35-c39f-4c0b-8576-cbd9882dd21d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:04.699640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919aa35-c39f-4c0b-8576-cbd9882dd21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:04.780307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f919aa35-c39f-4c0b-8576-cbd9882dd21d None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:04.781569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f919aa35-c39f-4c0b-8576-cbd9882dd21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:04.911450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 707/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Jul 15 13:54:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:54:04.926132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f23444-4d83-4a82-ab16-41a4d8d68806 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:04.926483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f23444-4d83-4a82-ab16-41a4d8d68806 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:04.926817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f23444-4d83-4a82-ab16-41a4d8d68806 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:04.928240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 705/1412] 10.222.0.16 () {54 vars in 918 bytes} [Fri Jul 15 13:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:04.934186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10128338-0925-4bae-96f7-f20c4343fdc3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:04.934474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10128338-0925-4bae-96f7-f20c4343fdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:04.934764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10128338-0925-4bae-96f7-f20c4343fdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:05.012288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:05.015140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-10128338-0925-4bae-96f7-f20c4343fdc3 None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:05.017011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-10128338-0925-4bae-96f7-f20c4343fdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:05.150232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 708/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Jul 15 13:54:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:07.744441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb75f17-1a06-4d56-849b-37f079160624 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:07.745023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb75f17-1a06-4d56-849b-37f079160624 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:07.745023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb75f17-1a06-4d56-849b-37f079160624 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:07.746408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:07.752091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8a7e7-a7f2-4901-9c7f-258bfc2ecad6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:07.752874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8a7e7-a7f2-4901-9c7f-258bfc2ecad6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:07.752874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8a7e7-a7f2-4901-9c7f-258bfc2ecad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:07.831427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fca8a7e7-a7f2-4901-9c7f-258bfc2ecad6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:07.832706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fca8a7e7-a7f2-4901-9c7f-258bfc2ecad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:07.894518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:07.941864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:07.989648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb75f17-1a06-4d56-849b-37f079160624 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:07.992145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6cb75f17-1a06-4d56-849b-37f079160624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:08.166829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb75f17-1a06-4d56-849b-37f079160624 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:08.243420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:08.285832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627479a0-271b-446d-aafa-78916242db8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:08.286487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627479a0-271b-446d-aafa-78916242db8b service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:08.286770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627479a0-271b-446d-aafa-78916242db8b service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:08.287047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627479a0-271b-446d-aafa-78916242db8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:08.288768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-627479a0-271b-446d-aafa-78916242db8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:08.381228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627479a0-271b-446d-aafa-78916242db8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:08.382850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627479a0-271b-446d-aafa-78916242db8b service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:08.401275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 707/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:09.325091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885de80-a040-48fc-a215-e5a02cf979b0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:09.325656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885de80-a040-48fc-a215-e5a02cf979b0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:09.325656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885de80-a040-48fc-a215-e5a02cf979b0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:09.327052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:09.332512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac85e-de29-4da9-a3fa-577c878117fd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:09.333089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac85e-de29-4da9-a3fa-577c878117fd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:09.333089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bac85e-de29-4da9-a3fa-577c878117fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:09.419933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-95bac85e-de29-4da9-a3fa-577c878117fd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:09.421115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-95bac85e-de29-4da9-a3fa-577c878117fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:09.450021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:09.527805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:09.682807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885de80-a040-48fc-a215-e5a02cf979b0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:09.684267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a885de80-a040-48fc-a215-e5a02cf979b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:09.842958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885de80-a040-48fc-a215-e5a02cf979b0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:09.886486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:09.967056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:09.967602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:09.967890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:09.968169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:09.969910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:10.082527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:10.084164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554f71c-6e85-4f16-8141-5878a8d20d34 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:10.105475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 711/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:11.060003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d51b0-b7a8-44ff-b877-ae227753b559 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:11.060611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d51b0-b7a8-44ff-b877-ae227753b559 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:11.061060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d51b0-b7a8-44ff-b877-ae227753b559 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:11.063744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:11.069300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615a38-83e4-40b6-bdbe-7a5160f4e5c8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:11.069833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615a38-83e4-40b6-bdbe-7a5160f4e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:11.070472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615a38-83e4-40b6-bdbe-7a5160f4e5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:11.128452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:11.156632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-48615a38-83e4-40b6-bdbe-7a5160f4e5c8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:11.158227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48615a38-83e4-40b6-bdbe-7a5160f4e5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:11.267651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:11.424721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d51b0-b7a8-44ff-b877-ae227753b559 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:11.426153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d53d51b0-b7a8-44ff-b877-ae227753b559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:11.579453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d51b0-b7a8-44ff-b877-ae227753b559 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:11.592718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:11.706791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:11.707356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:11.707637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:11.707912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:11.709638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:11.807386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:11.809030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9bb55-7b6e-4649-adac-3ff0122ee8fc service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:11.828638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 710/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:12.706667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b57626-b5a2-4976-b3dd-6a04a14f3f64 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:12.707240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b57626-b5a2-4976-b3dd-6a04a14f3f64 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:12.707533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b57626-b5a2-4976-b3dd-6a04a14f3f64 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:12.709019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:12.720281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5ce9b-ab32-4cde-8755-2dbb394fe744 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:12.720921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5ce9b-ab32-4cde-8755-2dbb394fe744 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:12.720921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5ce9b-ab32-4cde-8755-2dbb394fe744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:12.741067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:12.806321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a5e5ce9b-ab32-4cde-8755-2dbb394fe744 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:12.807569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a5e5ce9b-ab32-4cde-8755-2dbb394fe744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:12.914167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:13.065978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b57626-b5a2-4976-b3dd-6a04a14f3f64 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:13.067582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43b57626-b5a2-4976-b3dd-6a04a14f3f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:13.191098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:13.229443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b57626-b5a2-4976-b3dd-6a04a14f3f64 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:13.369034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:13.369608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:13.369887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:13.370160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:13.371821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:13.450011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:13.464676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:13.466297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc60169-7abc-4973-97a7-1eaba0d06bc8 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:13.487442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 714/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:14.367496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d204eb2-d74c-4140-a8dc-c767134f134b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:14.368033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d204eb2-d74c-4140-a8dc-c767134f134b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:14.368033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d204eb2-d74c-4140-a8dc-c767134f134b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:14.369482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:14.375017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e60ab-e918-46af-80e0-40330866dd60 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:14.375533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e60ab-e918-46af-80e0-40330866dd60 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:14.375643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e60ab-e918-46af-80e0-40330866dd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:14.455346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f49e60ab-e918-46af-80e0-40330866dd60 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:14.456501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f49e60ab-e918-46af-80e0-40330866dd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:14.566720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:14.716780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d204eb2-d74c-4140-a8dc-c767134f134b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:14.718221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5d204eb2-d74c-4140-a8dc-c767134f134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:14.808922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:14.878879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d204eb2-d74c-4140-a8dc-c767134f134b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:14.995086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:14.995655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:14.995935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:14.996219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:14.997919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:15.047885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:15.091870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:15.093515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90fc5f-f215-42fb-b746-8cc2e0d5bb7e service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:15.114108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 713/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:15.936756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac147b1-432f-45d8-a606-fac51a43202e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:15.937357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac147b1-432f-45d8-a606-fac51a43202e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:15.937357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac147b1-432f-45d8-a606-fac51a43202e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:15.939004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:54:15.945707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927aed5-c5e8-4a89-9b11-355cc0322449 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:15.946279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927aed5-c5e8-4a89-9b11-355cc0322449 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:15.946279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927aed5-c5e8-4a89-9b11-355cc0322449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:16.037047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6927aed5-c5e8-4a89-9b11-355cc0322449 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:16.038217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6927aed5-c5e8-4a89-9b11-355cc0322449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:16.162531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:16.316311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac147b1-432f-45d8-a606-fac51a43202e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:16.317932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7ac147b1-432f-45d8-a606-fac51a43202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:16.371107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:16.474576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac147b1-432f-45d8-a606-fac51a43202e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:16.721592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:16.722238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:16.722507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:16.722776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:16.724426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:16.732355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:16.818159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:16.819754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc77573-4bfd-4638-abe1-3769ec16b656 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:16.838560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 717/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:17.673502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee434b8-a502-4b15-a6e1-b5a97a8cecd9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:17.674074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee434b8-a502-4b15-a6e1-b5a97a8cecd9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:17.674074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee434b8-a502-4b15-a6e1-b5a97a8cecd9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:17.675492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:54:17.681700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f1212-b734-4094-b353-d9008a84fe01 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:17.682441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f1212-b734-4094-b353-d9008a84fe01 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:17.682610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f1212-b734-4094-b353-d9008a84fe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:17.766435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e35f1212-b734-4094-b353-d9008a84fe01 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:17.767664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e35f1212-b734-4094-b353-d9008a84fe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:17.847475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:17.883371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:18.033995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee434b8-a502-4b15-a6e1-b5a97a8cecd9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:18.035497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1ee434b8-a502-4b15-a6e1-b5a97a8cecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:18.055200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:18.200114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee434b8-a502-4b15-a6e1-b5a97a8cecd9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:18.304963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:18.332820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:18.333483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:18.333758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:18.334031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:18.336272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:18.462491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:18.465752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16932ac-776c-4763-b1a6-8eb1daea77ef service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:18.495407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 716/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:19.420829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdba85-dce1-417a-b995-6b877bbf5d27 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:19.421546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdba85-dce1-417a-b995-6b877bbf5d27 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:19.421546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdba85-dce1-417a-b995-6b877bbf5d27 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:19.422650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:19.431746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2b005-b800-481f-9ed2-aa1df7793682 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:19.432286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2b005-b800-481f-9ed2-aa1df7793682 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:19.432818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2b005-b800-481f-9ed2-aa1df7793682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:19.527477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eab2b005-b800-481f-9ed2-aa1df7793682 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:19.528741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eab2b005-b800-481f-9ed2-aa1df7793682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:19.571763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:19.634063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:19.791026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdba85-dce1-417a-b995-6b877bbf5d27 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:19.792593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-39fdba85-dce1-417a-b995-6b877bbf5d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:19.946554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdba85-dce1-417a-b995-6b877bbf5d27 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:20.024780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:20.094362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:20.095098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:20.095422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:20.095744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:20.097712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:20.210271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:20.213218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c83924-8e3d-40ab-b17b-88d3ecc195ee service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:20.242343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 720/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 13:54:21.123581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2470d-20a1-4cee-81e8-36bfca478ca5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:21.124203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2470d-20a1-4cee-81e8-36bfca478ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:21.124203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2470d-20a1-4cee-81e8-36bfca478ca5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:21.125821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:21.133823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98911f6-79e8-4c06-9720-e8f12ebf2c46 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:21.134629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98911f6-79e8-4c06-9720-e8f12ebf2c46 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:21.134629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98911f6-79e8-4c06-9720-e8f12ebf2c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:21.220383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c98911f6-79e8-4c06-9720-e8f12ebf2c46 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:21.221662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c98911f6-79e8-4c06-9720-e8f12ebf2c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:21.223872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:21.340889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:21.495962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2470d-20a1-4cee-81e8-36bfca478ca5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:21.497460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ac2470d-20a1-4cee-81e8-36bfca478ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:21.654138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2470d-20a1-4cee-81e8-36bfca478ca5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:21.682363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:21.787714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:21.788509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:21.788977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:21.789399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:21.792773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:21.927479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:21.929692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ba920-e601-442d-92b6-41390cb1cc8b service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:21.950343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 719/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 13:54:24.200280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbe83b-7254-4171-8114-344411a9deb8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:24.200877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbe83b-7254-4171-8114-344411a9deb8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:24.201140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbe83b-7254-4171-8114-344411a9deb8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:24.203384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:24.209588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4ab46-6fc5-4225-8a8d-31f62933479c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:24.210113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4ab46-6fc5-4225-8a8d-31f62933479c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:24.210987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4ab46-6fc5-4225-8a8d-31f62933479c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:24.277073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:24.320691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dbd4ab46-6fc5-4225-8a8d-31f62933479c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:24.322676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dbd4ab46-6fc5-4225-8a8d-31f62933479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:24.451559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:24.491131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbe83b-7254-4171-8114-344411a9deb8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:24.493270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4dbe83b-7254-4171-8114-344411a9deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:24.626273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbe83b-7254-4171-8114-344411a9deb8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:24.685928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:24.751159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:24.752007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:24.752528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:24.753019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:24.755183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:24.906238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:24.907716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaee77-efc6-4946-80ff-fca1856f7e85 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:24.929886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 723/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:27.175697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faac857-fe2c-4c23-bf04-02fee2a12b4e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:27.176291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faac857-fe2c-4c23-bf04-02fee2a12b4e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:27.176291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faac857-fe2c-4c23-bf04-02fee2a12b4e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:27.177684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:27.184313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4d9e2-cb3e-4026-b3d3-852d1d115bbc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:27.185117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4d9e2-cb3e-4026-b3d3-852d1d115bbc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:27.185323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4d9e2-cb3e-4026-b3d3-852d1d115bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:27.257585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b5f4d9e2-cb3e-4026-b3d3-852d1d115bbc None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:27.258817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b5f4d9e2-cb3e-4026-b3d3-852d1d115bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:27.272203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:27.364907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:27.405544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faac857-fe2c-4c23-bf04-02fee2a12b4e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:27.407094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9faac857-fe2c-4c23-bf04-02fee2a12b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:27.522619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faac857-fe2c-4c23-bf04-02fee2a12b4e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:27.538156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:27.613093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:27.613764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:27.614048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:27.614328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:27.616142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:27.734848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:27.736306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c360a9-8a7e-4724-b625-7d1188aadbdf service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:27.755397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 722/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 13:54:30.265640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059b117-3e15-4f78-85b9-13e814e77fa4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:30.266242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059b117-3e15-4f78-85b9-13e814e77fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:30.266344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059b117-3e15-4f78-85b9-13e814e77fa4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:30.268360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:30.276650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e097b-7668-4838-aea1-6f40afc5a48e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:30.277371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e097b-7668-4838-aea1-6f40afc5a48e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:30.277541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e097b-7668-4838-aea1-6f40afc5a48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:30.339731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:30.362267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-525e097b-7668-4838-aea1-6f40afc5a48e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:30.363524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-525e097b-7668-4838-aea1-6f40afc5a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:30.525310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:30.564662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059b117-3e15-4f78-85b9-13e814e77fa4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:30.566200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c059b117-3e15-4f78-85b9-13e814e77fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:30.681649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059b117-3e15-4f78-85b9-13e814e77fa4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:30.727736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:30.766056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:30.766711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:30.766992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:30.767271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:30.768994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:30.905309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:30.906767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b24ae6-3c45-4830-af66-68ee8bec92f0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:30.925620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 726/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 13:54:33.149654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413f32c-d63e-4b73-8f30-8b98fe24b217 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:33.150469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413f32c-d63e-4b73-8f30-8b98fe24b217 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:33.150693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413f32c-d63e-4b73-8f30-8b98fe24b217 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:33.152749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:33.160313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872d3-1d4f-4003-8143-062cf2a0cb0c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:33.160313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872d3-1d4f-4003-8143-062cf2a0cb0c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:33.160993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872d3-1d4f-4003-8143-062cf2a0cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:33.239366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f63872d3-1d4f-4003-8143-062cf2a0cb0c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:33.240641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f63872d3-1d4f-4003-8143-062cf2a0cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:33.265551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:33.355933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:33.395779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413f32c-d63e-4b73-8f30-8b98fe24b217 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:33.397381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9413f32c-d63e-4b73-8f30-8b98fe24b217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:33.510011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9413f32c-d63e-4b73-8f30-8b98fe24b217 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:33.530843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:33.587795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:33.588497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:33.588838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:33.589049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:33.590761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:33.706414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:33.707875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a0de9-e5a2-411a-8f0e-8e072795fc4c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:33.727743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 725/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:35.831414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d0098-2400-4e88-91e1-6be2aa14a740 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:35.831991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d0098-2400-4e88-91e1-6be2aa14a740 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:35.831991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d0098-2400-4e88-91e1-6be2aa14a740 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:35.833421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:35.839976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614e10e-f2dc-42c0-b4af-058aca7d5219 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:35.840910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614e10e-f2dc-42c0-b4af-058aca7d5219 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:35.841087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614e10e-f2dc-42c0-b4af-058aca7d5219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:35.910395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:35.913539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2614e10e-f2dc-42c0-b4af-058aca7d5219 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:35.916589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2614e10e-f2dc-42c0-b4af-058aca7d5219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:36.033615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:36.085422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d0098-2400-4e88-91e1-6be2aa14a740 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:36.088930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d35d0098-2400-4e88-91e1-6be2aa14a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:36.232916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d0098-2400-4e88-91e1-6be2aa14a740 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:36.286117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:36.322485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:36.323648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:36.324175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:36.324732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:36.328074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:36.461323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:36.462722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dca1c7-7ae9-4a27-8049-61b6f1ee0928 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:36.484915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 729/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:38.667101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f11d3-c821-446e-a6b0-fa9a69c1a3e0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:38.667699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f11d3-c821-446e-a6b0-fa9a69c1a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:38.667699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f11d3-c821-446e-a6b0-fa9a69c1a3e0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:38.669081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:38.674421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1db57-b23a-4797-97a9-4ef6b5780b66 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:38.674886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1db57-b23a-4797-97a9-4ef6b5780b66 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:38.674993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1db57-b23a-4797-97a9-4ef6b5780b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:38.745652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c6e1db57-b23a-4797-97a9-4ef6b5780b66 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:38.746922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c6e1db57-b23a-4797-97a9-4ef6b5780b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:38.779208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:38.856947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:38.896691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f11d3-c821-446e-a6b0-fa9a69c1a3e0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:38.898274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fb2f11d3-c821-446e-a6b0-fa9a69c1a3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:39.014407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f11d3-c821-446e-a6b0-fa9a69c1a3e0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:39.069867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:39.151714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:39.152895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:39.153412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:39.153938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:39.157327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:39.316553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:39.318009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a9e99-f0ee-46d5-85a4-5276ce09dcba service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:39.337262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 728/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:41.411932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e9f5d-e992-471c-8529-b054edab99e2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:41.412491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e9f5d-e992-471c-8529-b054edab99e2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:41.412491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e9f5d-e992-471c-8529-b054edab99e2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:41.414083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:41.419343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effaee28-d4d7-4051-9f61-51abe6088d4e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:41.419807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effaee28-d4d7-4051-9f61-51abe6088d4e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:41.419908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effaee28-d4d7-4051-9f61-51abe6088d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:41.495527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-effaee28-d4d7-4051-9f61-51abe6088d4e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:41.496805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-effaee28-d4d7-4051-9f61-51abe6088d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:41.505129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:41.599966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:41.640537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e9f5d-e992-471c-8529-b054edab99e2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:41.642182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cc7e9f5d-e992-471c-8529-b054edab99e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:41.750350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e9f5d-e992-471c-8529-b054edab99e2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:41.810256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:41.830236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:41.830890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:41.831180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:41.831458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:41.833191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:41.946343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:41.947792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94ccc5-9506-4adb-bb8c-e585a225da21 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:41.966415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 732/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:44.080333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e525a3-6ba4-440f-99e9-81ec5cf9a2e7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:44.080936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e525a3-6ba4-440f-99e9-81ec5cf9a2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:44.080936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e525a3-6ba4-440f-99e9-81ec5cf9a2e7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:44.082309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:44.094236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9a375-3542-499d-90d2-9631dad77e72 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:44.094807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9a375-3542-499d-90d2-9631dad77e72 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:44.094940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9a375-3542-499d-90d2-9631dad77e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:44.170372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dde9a375-3542-499d-90d2-9631dad77e72 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:44.171641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dde9a375-3542-499d-90d2-9631dad77e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:44.211886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:44.283726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:54:44.325457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e525a3-6ba4-440f-99e9-81ec5cf9a2e7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:44.327020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-73e525a3-6ba4-440f-99e9-81ec5cf9a2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:44.438714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e525a3-6ba4-440f-99e9-81ec5cf9a2e7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:44.478914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:44.516332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:44.517009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:44.517291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:44.517566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:44.519206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:44.636022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:44.637714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a1a5-cde8-43dd-a44f-8dfa5de97fce service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:44.657880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 731/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:46.735418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955ae38-3144-4437-aede-42df5159ea11 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:46.735971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955ae38-3144-4437-aede-42df5159ea11 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:46.735971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955ae38-3144-4437-aede-42df5159ea11 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:46.737398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:46.742933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8384c-e5ea-43ef-a170-0b23ef479db3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:46.743374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8384c-e5ea-43ef-a170-0b23ef479db3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:46.743500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8384c-e5ea-43ef-a170-0b23ef479db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:46.815919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-52b8384c-e5ea-43ef-a170-0b23ef479db3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:46.817197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-52b8384c-e5ea-43ef-a170-0b23ef479db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:46.836237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:46.917780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:46.954053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955ae38-3144-4437-aede-42df5159ea11 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:46.955510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6955ae38-3144-4437-aede-42df5159ea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:47.060952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955ae38-3144-4437-aede-42df5159ea11 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:47.129517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:47.138371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:47.139042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:47.139326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:47.139603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:47.141460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:47.267174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:47.268680 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb97bd9-2e91-4972-9178-27fecf67ae5e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:47.288864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 735/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:49.361157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775df05-caff-4bd9-a725-5666b1b2313f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:49.361723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775df05-caff-4bd9-a725-5666b1b2313f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:49.361723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775df05-caff-4bd9-a725-5666b1b2313f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:49.363225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:54:49.369292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4fbe3-7a80-4528-b225-401000f556b4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:49.369745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4fbe3-7a80-4528-b225-401000f556b4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:49.369890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4fbe3-7a80-4528-b225-401000f556b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:49.437370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a2c4fbe3-7a80-4528-b225-401000f556b4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:54:49.438590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a2c4fbe3-7a80-4528-b225-401000f556b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:49.487729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:49.536965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:54:49.574303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775df05-caff-4bd9-a725-5666b1b2313f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:54:49.575855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e775df05-caff-4bd9-a725-5666b1b2313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:49.682368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775df05-caff-4bd9-a725-5666b1b2313f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:54:49.731696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:49.760445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:54:49.761115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:54:49.761406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:54:49.761688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:54:49.763367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:54:49.875335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:54:49.876818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d231077-8bbc-4046-876b-ced2d18acf87 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:54:49.894952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 734/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:54:59.535190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=22 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:54:59.892627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=66 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:09.648762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18715b6d-5ca6-471a-9931-edcf56b4812b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:09.649302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18715b6d-5ca6-471a-9931-edcf56b4812b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:09.649302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18715b6d-5ca6-471a-9931-edcf56b4812b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:09.650707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 737/1471] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:09.656072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6937-7ff1-475d-b3ed-8e7236ab8128 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:09.656552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6937-7ff1-475d-b3ed-8e7236ab8128 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:09.656662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec6937-7ff1-475d-b3ed-8e7236ab8128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:09.722181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-09ec6937-7ff1-475d-b3ed-8e7236ab8128 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:09.723408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-09ec6937-7ff1-475d-b3ed-8e7236ab8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:09.822470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 735/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:09.870957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2c272-13f2-4461-898d-0d37395ee30c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:09.871265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2c272-13f2-4461-898d-0d37395ee30c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:09.871538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2c272-13f2-4461-898d-0d37395ee30c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:09.872966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 738/1473] 10.222.0.16 () {54 vars in 960 bytes} [Fri Jul 15 13:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:09.877919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb510499-4a2c-4aee-ac23-3a2e28385735 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:09.878213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb510499-4a2c-4aee-ac23-3a2e28385735 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:09.878491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb510499-4a2c-4aee-ac23-3a2e28385735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:09.954444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fb510499-4a2c-4aee-ac23-3a2e28385735 None None] MFA Rules not processed for user `deb87accdcab4759add4b76958d151a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:09.955588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fb510499-4a2c-4aee-ac23-3a2e28385735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:09.976476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:10.034540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 736/1474] 10.222.0.16 () {58 vars in 1057 bytes} [Fri Jul 15 13:55:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:10.042216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2c272-13f2-4461-898d-0d37395ee30c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:10.043756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dfa2c272-13f2-4461-898d-0d37395ee30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:10.148231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2c272-13f2-4461-898d-0d37395ee30c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:10.222080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:10.222651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:10.222939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:10.223219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:10.225014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:10.280448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:10.337925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:10.339372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b62dc1-3085-4da4-a1e5-074ee4c479a4 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:10.358604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 739/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:55:12.526071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3038ad-6693-4865-882d-ce2ebfe5cfb1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:12.526623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3038ad-6693-4865-882d-ce2ebfe5cfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:12.526623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3038ad-6693-4865-882d-ce2ebfe5cfb1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:12.527938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 737/1476] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:12.533259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7581e-c375-4868-8a97-b3d74f1438bd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:12.533706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7581e-c375-4868-8a97-b3d74f1438bd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:12.533840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7581e-c375-4868-8a97-b3d74f1438bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:12.599738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-90a7581e-c375-4868-8a97-b3d74f1438bd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:12.601330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-90a7581e-c375-4868-8a97-b3d74f1438bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:12.703401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 740/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:12.727302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3038ad-6693-4865-882d-ce2ebfe5cfb1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:12.728886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7c3038ad-6693-4865-882d-ce2ebfe5cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:12.867103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3038ad-6693-4865-882d-ce2ebfe5cfb1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:12.917377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:12.966201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:12.966864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:12.967149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:12.967426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:12.968158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:12.969977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:55:12.969977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 13:55:12.969977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 13:55:12.971125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13cb56-f6e7-4ddf-922e-daf2faec1b14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:12.990310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 738/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jul 15 13:55:12 2022] GET /identity/v3/projects => generated 2553 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 13:55:15.075741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d99f1-c050-4117-bf00-2ca05070efc4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:15.076302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d99f1-c050-4117-bf00-2ca05070efc4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:15.076302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d99f1-c050-4117-bf00-2ca05070efc4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:15.077769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 741/1479] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:15.083103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a9788-66c4-454d-b42e-ff9980c603a5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:15.083558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a9788-66c4-454d-b42e-ff9980c603a5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:15.083657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a9788-66c4-454d-b42e-ff9980c603a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:15.153917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-177a9788-66c4-454d-b42e-ff9980c603a5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:15.155141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-177a9788-66c4-454d-b42e-ff9980c603a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:15.260784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 739/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:15.304515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d99f1-c050-4117-bf00-2ca05070efc4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:15.306614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ef1d99f1-c050-4117-bf00-2ca05070efc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:15.317490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:15.454733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d99f1-c050-4117-bf00-2ca05070efc4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:15.536679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:15.570437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:15.571087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:15.571366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:15.571636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:15.573324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:15.664057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:15.665700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd8439-f745-404c-8899-2723cb60bbd3 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:15.683687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 742/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:55:17.907757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea066-dd5c-4264-8d5a-4926f3f4999f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:17.908309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea066-dd5c-4264-8d5a-4926f3f4999f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:17.908309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea066-dd5c-4264-8d5a-4926f3f4999f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:17.909713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 740/1482] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:17.914921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9577b-11dd-4238-86ee-8fe04aa3bf96 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:17.915355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9577b-11dd-4238-86ee-8fe04aa3bf96 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:17.915463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9577b-11dd-4238-86ee-8fe04aa3bf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:17.981607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c4c9577b-11dd-4238-86ee-8fe04aa3bf96 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:17.982830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c4c9577b-11dd-4238-86ee-8fe04aa3bf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:18.029822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:18.082372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 743/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:18.119596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea066-dd5c-4264-8d5a-4926f3f4999f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:18.121163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6c8ea066-dd5c-4264-8d5a-4926f3f4999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:18.149470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:18.231121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea066-dd5c-4264-8d5a-4926f3f4999f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:18.305725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:18.306408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:18.306696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:18.306977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:18.308731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:18.385682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:18.423799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:18.425643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53deb5e9-d799-496d-9729-ff531c28f3d6 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:18.450859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 741/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:55:20.577024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455abb28-27e0-4066-9273-07309827c199 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:20.577628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455abb28-27e0-4066-9273-07309827c199 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:20.578206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455abb28-27e0-4066-9273-07309827c199 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:20.579836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 744/1485] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:20.586801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b0a7b-3c4a-47a1-9e6b-7954555a3ca2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:20.587290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b0a7b-3c4a-47a1-9e6b-7954555a3ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:20.587384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b0a7b-3c4a-47a1-9e6b-7954555a3ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:20.667072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-538b0a7b-3c4a-47a1-9e6b-7954555a3ca2 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:20.668914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-538b0a7b-3c4a-47a1-9e6b-7954555a3ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:20.785904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 742/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:20.826004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455abb28-27e0-4066-9273-07309827c199 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:20.827596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-455abb28-27e0-4066-9273-07309827c199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:20.945463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455abb28-27e0-4066-9273-07309827c199 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:21.023521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:21.025377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:21.026052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:21.026545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:21.027002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:21.028944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:21.154900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:21.156275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912d59e-81ff-4a10-8701-9835758c6407 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:21.177978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 745/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:55:23.484173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294e2d-ae31-40d6-8dcf-7d3ba40f6425 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:23.484745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294e2d-ae31-40d6-8dcf-7d3ba40f6425 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:23.484745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74294e2d-ae31-40d6-8dcf-7d3ba40f6425 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:23.486040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 743/1488] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:23.491720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f321c-533f-4555-831c-431fd6b6a67e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:23.492169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f321c-533f-4555-831c-431fd6b6a67e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:23.492306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f321c-533f-4555-831c-431fd6b6a67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:23.557267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3e4f321c-533f-4555-831c-431fd6b6a67e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:23.558509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3e4f321c-533f-4555-831c-431fd6b6a67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:23.612737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:23.658937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 746/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:55:23.699693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294e2d-ae31-40d6-8dcf-7d3ba40f6425 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:23.701288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-74294e2d-ae31-40d6-8dcf-7d3ba40f6425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:23.739200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:23.808080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74294e2d-ae31-40d6-8dcf-7d3ba40f6425 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:23.883830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:23.884510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:23.884797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:23.885079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:23.886736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:23.968295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:23.997831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:23.999198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf937f67-1f83-40ab-89ee-76dedf45475d service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:24.017591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 744/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:55:26.235268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0b329-f0e9-4344-b14c-f4e0674a02ff None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:26.235847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0b329-f0e9-4344-b14c-f4e0674a02ff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:26.235955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0b329-f0e9-4344-b14c-f4e0674a02ff None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:26.237317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 747/1491] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jul 15 13:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:55:26.243431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3932a-6b50-4ffe-8d1a-fc7c7cb12739 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:26.243860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3932a-6b50-4ffe-8d1a-fc7c7cb12739 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:26.243972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3932a-6b50-4ffe-8d1a-fc7c7cb12739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:26.313130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f9e3932a-6b50-4ffe-8d1a-fc7c7cb12739 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:55:26.314283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f9e3932a-6b50-4ffe-8d1a-fc7c7cb12739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:26.533786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 745/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jul 15 13:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:55:26.580520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0b329-f0e9-4344-b14c-f4e0674a02ff None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:55:26.582701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29c0b329-f0e9-4344-b14c-f4e0674a02ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:26.692699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0b329-f0e9-4344-b14c-f4e0674a02ff None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:55:26.768154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:55:26.768831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:55:26.769113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:55:26.769388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:55:26.771033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:55:26.780849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:26.885173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:55:26.886535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7457266-8ce3-492e-8edf-b01b86992865 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:55:26.905493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 748/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:55:36.531972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=82 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:55:36.903027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=50 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:09.605668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:09.606292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:09.606292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:09.679190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:57:09.680424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:09.777011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 746/1494] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:57:09.784985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:09.785505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:09.785613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:09.861586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:57:09.862734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:09.961657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 749/1495] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:57:09.966465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:09.967679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:09.979394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3491c8f-dfbc-451d-8691-57811831e1b1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:10.041211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:10.076069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:10.076767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.077050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.077331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:10.078050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 13:57:10.078355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:10.079365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:10.079365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 13:57:10.079365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 13:57:10.080247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:10.092098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 747/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 13:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:57:10.100219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:10.109796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100504da-ca85-4bc2-9c79-8eea23ff1482 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:10.174380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:10.211128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:10.211784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.212062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.212333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:10.213047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:10.214787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:10.214787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 13:57:10.214787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 13:57:10.215983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:10.272645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 750/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 13:57:10 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:57:10.276842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:10.278408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:10.314859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117e34c-e4cd-4da3-afcd-8c8a65711ea9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:10.412468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:10.413200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.413427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.413701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:10.414422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:10.415411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:10.415411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 13:57:10.415411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 13:57:10.416651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:10.458663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 748/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 13:57:10 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 13:57:10.462909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:10.464457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:10.527508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651267eb-2d3a-40bd-989a-19df69b4efcd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:10.613169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:10.625592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:10.626301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.626574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.626846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:10.627573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:10.628603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:10.628603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 13:57:10.628603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 13:57:10.629498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:10.641961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 751/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 13:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 13:57:10.646095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:10.647614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:10.659065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2eea3-64be-42f8-9c79-e405c998baa1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:10.684949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:10.756950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:10.757621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.757907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.758183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:10.758921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:10.759937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:10.759937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 13:57:10.759937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 13:57:10.760849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fd36a-93d8-4cfa-b7f7-15d84e0e6526 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:10.774145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 749/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 13:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 13:57:10.778436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:10.779976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:10.815167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ced80-8411-4615-98f4-c45fbfb2a9df admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:10.911291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:10.912194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.912499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.912775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] PATH_INFO: `/v3/projects/65a9e933d99648d198cad2a5a0c87b56/users/e9b2a56c5959488797daca8dd5ac4e59/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:10.935929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a9e933d99648d198cad2a5a0c87b56, user_id=e9b2a56c5959488797daca8dd5ac4e59, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:10.936916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:10.936916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 13:57:10.936916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 13:57:10.938948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02eaef3-a5e9-4d00-acdc-d94d7e0225dd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:10.968169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 752/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 13:57:10 2022] PUT /identity/v3/projects/65a9e933d99648d198cad2a5a0c87b56/users/e9b2a56c5959488797daca8dd5ac4e59/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:57:10.981208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc01f6-030e-4894-bc9b-ba9cd2088843 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:10.981497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc01f6-030e-4894-bc9b-ba9cd2088843 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:10.981803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc01f6-030e-4894-bc9b-ba9cd2088843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:11.037985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-abcc01f6-030e-4894-bc9b-ba9cd2088843 None None] MFA Rules not processed for user `e9b2a56c5959488797daca8dd5ac4e59`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:57:11.039258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-abcc01f6-030e-4894-bc9b-ba9cd2088843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:11.117208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 750/1502] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 13:57:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:57:11.130997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66b3b-670a-4077-b02e-0e5822f6a07f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:11.131392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66b3b-670a-4077-b02e-0e5822f6a07f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:11.131745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66b3b-670a-4077-b02e-0e5822f6a07f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:11.133232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 753/1503] 10.222.0.16 () {54 vars in 963 bytes} [Fri Jul 15 13:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:57:11.138298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759271ad-c297-4881-bd8f-300f53ff4c12 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:11.138590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759271ad-c297-4881-bd8f-300f53ff4c12 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:11.138874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759271ad-c297-4881-bd8f-300f53ff4c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:11.170202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:11.216495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-759271ad-c297-4881-bd8f-300f53ff4c12 None None] MFA Rules not processed for user `f0e94eecf28f42468e14bbd0c4eb3777`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:57:11.217613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-759271ad-c297-4881-bd8f-300f53ff4c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:11.300567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 751/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Fri Jul 15 13:57:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 13:57:11.308564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66b3b-670a-4077-b02e-0e5822f6a07f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:11.310130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29d66b3b-670a-4077-b02e-0e5822f6a07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:11.361234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:11.417705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66b3b-670a-4077-b02e-0e5822f6a07f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:11.491625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:11.492179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:11.492485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:11.492757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:11.494393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:11.599536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:11.601052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:11.607359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:11.627992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 754/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:57:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:57:11.671116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac25e46-efb6-4800-bb63-a5365b5e695b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:11.671569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac25e46-efb6-4800-bb63-a5365b5e695b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:11.671684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac25e46-efb6-4800-bb63-a5365b5e695b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:11.672969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 752/1506] 10.222.0.16 () {52 vars in 829 bytes} [Fri Jul 15 13:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:57:11.677243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:11.678468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:11.781180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d86ee-bc5a-4901-9bcc-6672bb68c114 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:11.878046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:11.878922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:11.879219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:11.879495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] PATH_INFO: `/v3/users/e9b2a56c5959488797daca8dd5ac4e59` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:11.892713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b2a56c5959488797daca8dd5ac4e59)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:11.893744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:11.893744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 13:57:11.893744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 13:57:11.894987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:11.904842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:11.961933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b2a56c5959488797daca8dd5ac4e59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:11.980841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b2a56c5959488797daca8dd5ac4e59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:11.995633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b2a56c5959488797daca8dd5ac4e59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:12.005325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b2a56c5959488797daca8dd5ac4e59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:12.005853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b2a56c5959488797daca8dd5ac4e59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:12.011855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 755/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 13:57:11 2022] DELETE /identity/v3/users/e9b2a56c5959488797daca8dd5ac4e59 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:57:12.016480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:12.016769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:12.017050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:12.075010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:57:12.076223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:12.157150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:12.173012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 753/1508] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 13:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:57:12.183308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:12.184581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:12.287750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dda37-f3b8-4874-ae98-f8250affac2d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:12.344156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:12.366138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:12.366796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:12.367083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:12.367446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:12.369080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:12.481308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:12.482781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2647dd-fdc5-4d7c-ba76-eb07adf8fef1 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:12.502199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 756/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 13:57:12.847374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:57:12.848774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:57:12.860468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ce521-f4ef-4422-bd37-ee0936b911dd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:57:12.964648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:57:12.965516 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:57:12.965800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:57:12.966077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] PATH_INFO: `/v3/projects/65a9e933d99648d198cad2a5a0c87b56` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:57:12.976363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a9e933d99648d198cad2a5a0c87b56)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:57:12.977429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 13:57:12.977429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 13:57:12.977429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 13:57:12.978897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:57:13.039065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a9e933d99648d198cad2a5a0c87b56', 'request_id': 'req-ad084eb7-4e9a-411d-9803-cf00e2289bc8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:13.053372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a9e933d99648d198cad2a5a0c87b56', 'request_id': 'req-ad084eb7-4e9a-411d-9803-cf00e2289bc8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:13.065387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a9e933d99648d198cad2a5a0c87b56', 'request_id': 'req-ad084eb7-4e9a-411d-9803-cf00e2289bc8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:13.090235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] The token cache is being invalidate because project 65a9e933d99648d198cad2a5a0c87b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 13:57:13.090524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:13.091061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ad084eb7-4e9a-411d-9803-cf00e2289bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 13:57:13.092560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 754/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 13:57:12 2022] DELETE /identity/v3/projects/65a9e933d99648d198cad2a5a0c87b56 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 13:57:22.499891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=62 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:57:23.087959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:58:50.743149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322a8f-6130-409a-8b75-96ef0b065b26 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:50.743784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322a8f-6130-409a-8b75-96ef0b065b26 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:50.743784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322a8f-6130-409a-8b75-96ef0b065b26 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:50.745133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:58:50.749910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee97b1f-0c7c-481d-8d02-c6b21fd0a762 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:50.750321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee97b1f-0c7c-481d-8d02-c6b21fd0a762 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:50.750505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee97b1f-0c7c-481d-8d02-c6b21fd0a762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:50.831174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5ee97b1f-0c7c-481d-8d02-c6b21fd0a762 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:58:50.832375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5ee97b1f-0c7c-481d-8d02-c6b21fd0a762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:58:50.935755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:58:51.092974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2322a8f-6130-409a-8b75-96ef0b065b26 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:58:51.095180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2322a8f-6130-409a-8b75-96ef0b065b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:58:51.215613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2322a8f-6130-409a-8b75-96ef0b065b26 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:58:51.221122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37d80f-62c8-43be-bdbd-78e518a6978d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:58:51.221768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d80f-62c8-43be-bdbd-78e518a6978d service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:51.222055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d80f-62c8-43be-bdbd-78e518a6978d service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:51.222331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d80f-62c8-43be-bdbd-78e518a6978d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:51.231352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37d80f-62c8-43be-bdbd-78e518a6978d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:58:51.232966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37d80f-62c8-43be-bdbd-78e518a6978d service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:58:51.242365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 758/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:58:52.075250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e697dde-2c79-47ce-aad2-d4080b34a0ca None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:52.075833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e697dde-2c79-47ce-aad2-d4080b34a0ca None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:52.075833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e697dde-2c79-47ce-aad2-d4080b34a0ca None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:52.077312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:58:52.083003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2b18-094a-4329-964a-8162d91d72bb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:52.083502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2b18-094a-4329-964a-8162d91d72bb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:52.083502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2b18-094a-4329-964a-8162d91d72bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:52.101919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f33d2b18-094a-4329-964a-8162d91d72bb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:58:52.103127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f33d2b18-094a-4329-964a-8162d91d72bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:58:52.111364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:58:52.265379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e697dde-2c79-47ce-aad2-d4080b34a0ca None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:58:52.281124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e697dde-2c79-47ce-aad2-d4080b34a0ca None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:58:52.287440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de4f6c-1f64-45d1-ad4b-dc571049547c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:58:52.288559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de4f6c-1f64-45d1-ad4b-dc571049547c service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:52.288987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de4f6c-1f64-45d1-ad4b-dc571049547c service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:52.289276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de4f6c-1f64-45d1-ad4b-dc571049547c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:52.299637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de4f6c-1f64-45d1-ad4b-dc571049547c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:58:52.301327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de4f6c-1f64-45d1-ad4b-dc571049547c service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:58:52.311573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 757/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:58:52.612107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9749de2-1044-429f-bc40-e7ff5a13bc61 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:52.612459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9749de2-1044-429f-bc40-e7ff5a13bc61 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:52.612748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9749de2-1044-429f-bc40-e7ff5a13bc61 None None] PATH_INFO: `/v3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:52.614560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 760/1517] 10.222.0.64 () {54 vars in 925 bytes} [Fri Jul 15 13:58:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 13:58:52.627705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e447-da0f-4b9b-8fc9-8cfa3b5ef6fe None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:52.628080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e447-da0f-4b9b-8fc9-8cfa3b5ef6fe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:52.628448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e447-da0f-4b9b-8fc9-8cfa3b5ef6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:52.686991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b2b8e447-da0f-4b9b-8fc9-8cfa3b5ef6fe None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:58:52.688212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b2b8e447-da0f-4b9b-8fc9-8cfa3b5ef6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:58:52.705917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:58:52.761518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 758/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Jul 15 13:58:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:58:56.768244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62b37-24c5-4e2e-87c6-972f716bb4df None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:56.768848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62b37-24c5-4e2e-87c6-972f716bb4df None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:56.768988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b62b37-24c5-4e2e-87c6-972f716bb4df None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:56.770699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 761/1519] 10.222.0.16 () {54 vars in 960 bytes} [Fri Jul 15 13:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:58:56.776677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da907-8b28-48e7-8045-16a5796d3c6d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:56.777286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da907-8b28-48e7-8045-16a5796d3c6d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:56.777427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da907-8b28-48e7-8045-16a5796d3c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:56.796013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-425da907-8b28-48e7-8045-16a5796d3c6d None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:58:56.798439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-425da907-8b28-48e7-8045-16a5796d3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:58:56.807735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 759/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Fri Jul 15 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:58:56.816861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b62b37-24c5-4e2e-87c6-972f716bb4df None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:58:56.835724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b62b37-24c5-4e2e-87c6-972f716bb4df None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:58:56.842691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2517008-f02c-42ed-961c-35049ebc6046 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:58:56.843293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2517008-f02c-42ed-961c-35049ebc6046 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:58:56.843576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2517008-f02c-42ed-961c-35049ebc6046 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:58:56.843874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2517008-f02c-42ed-961c-35049ebc6046 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:58:56.853565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2517008-f02c-42ed-961c-35049ebc6046 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:58:56.855422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2517008-f02c-42ed-961c-35049ebc6046 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:58:56.865672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 762/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:58:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:03.544924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67da0c-ee4e-4328-9a71-0ac9523558b4 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:03.545453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67da0c-ee4e-4328-9a71-0ac9523558b4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:03.545453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67da0c-ee4e-4328-9a71-0ac9523558b4 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:03.546790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:59:03.551526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac653ae7-c52d-40b5-bff5-eb1666f1a4a3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:03.551960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac653ae7-c52d-40b5-bff5-eb1666f1a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:03.552089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac653ae7-c52d-40b5-bff5-eb1666f1a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:03.571607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ac653ae7-c52d-40b5-bff5-eb1666f1a4a3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:59:03.572837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ac653ae7-c52d-40b5-bff5-eb1666f1a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:03.580703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:59:03.736490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67da0c-ee4e-4328-9a71-0ac9523558b4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:59:03.751282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67da0c-ee4e-4328-9a71-0ac9523558b4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:59:03.756599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26219c3e-0b02-4b6f-b39d-498fdc45aa5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:59:03.757236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26219c3e-0b02-4b6f-b39d-498fdc45aa5a service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:03.757518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26219c3e-0b02-4b6f-b39d-498fdc45aa5a service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:03.757795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26219c3e-0b02-4b6f-b39d-498fdc45aa5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:03.766486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26219c3e-0b02-4b6f-b39d-498fdc45aa5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:59:03.768055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26219c3e-0b02-4b6f-b39d-498fdc45aa5a service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:59:03.777421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 761/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:04.612091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867c6c7-4ae3-4ea5-99e1-ae175b1c4ab6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:04.612691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867c6c7-4ae3-4ea5-99e1-ae175b1c4ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:04.612691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867c6c7-4ae3-4ea5-99e1-ae175b1c4ab6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:04.614094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 13:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:59:04.619341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44c24e-b29f-41e7-9f5c-66ff17e6778e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:04.619736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44c24e-b29f-41e7-9f5c-66ff17e6778e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:04.619899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44c24e-b29f-41e7-9f5c-66ff17e6778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:04.638201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5f44c24e-b29f-41e7-9f5c-66ff17e6778e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:59:04.639421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f44c24e-b29f-41e7-9f5c-66ff17e6778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:04.647344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 13:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:59:04.799263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867c6c7-4ae3-4ea5-99e1-ae175b1c4ab6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:59:04.814213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867c6c7-4ae3-4ea5-99e1-ae175b1c4ab6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:59:04.819691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a34cf-66a4-4316-b877-a162f7dad54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:59:04.820333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a34cf-66a4-4316-b877-a162f7dad54f service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:04.821235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a34cf-66a4-4316-b877-a162f7dad54f service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:04.821514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a34cf-66a4-4316-b877-a162f7dad54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:04.829947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a34cf-66a4-4316-b877-a162f7dad54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:59:04.831536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a34cf-66a4-4316-b877-a162f7dad54f service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:59:04.841355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 765/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:08.661165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d9d8-ea78-47ef-8cf9-298133a6440b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:08.661706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d9d8-ea78-47ef-8cf9-298133a6440b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:08.661706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d9d8-ea78-47ef-8cf9-298133a6440b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:08.663109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 13:59:08.668429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4287d09-5769-41b7-8bea-d6dc024018f7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:08.668863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4287d09-5769-41b7-8bea-d6dc024018f7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:08.668999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4287d09-5769-41b7-8bea-d6dc024018f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:08.687020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4287d09-5769-41b7-8bea-d6dc024018f7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:59:08.688220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4287d09-5769-41b7-8bea-d6dc024018f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:08.696624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:59:08.734239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d9d8-ea78-47ef-8cf9-298133a6440b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:59:08.735976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4867d9d8-ea78-47ef-8cf9-298133a6440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:08.813560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d9d8-ea78-47ef-8cf9-298133a6440b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:59:08.817115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe31880-8d43-48c5-bdad-f1e2ae57ee29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:59:08.817769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe31880-8d43-48c5-bdad-f1e2ae57ee29 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:08.818050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe31880-8d43-48c5-bdad-f1e2ae57ee29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:08.818327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe31880-8d43-48c5-bdad-f1e2ae57ee29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:08.827876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe31880-8d43-48c5-bdad-f1e2ae57ee29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:59:08.829340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe31880-8d43-48c5-bdad-f1e2ae57ee29 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:59:08.839649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 764/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:10.865855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cff7f-922e-43d1-a023-bfd413314879 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:10.866362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cff7f-922e-43d1-a023-bfd413314879 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:10.866476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cff7f-922e-43d1-a023-bfd413314879 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:10.868423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:59:10.873702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a6c71-516f-40de-bb5d-f91dad9351fd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:10.874128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a6c71-516f-40de-bb5d-f91dad9351fd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:10.874282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a6c71-516f-40de-bb5d-f91dad9351fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:10.892213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-420a6c71-516f-40de-bb5d-f91dad9351fd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:59:10.893429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-420a6c71-516f-40de-bb5d-f91dad9351fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:10.901209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:59:10.937474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cff7f-922e-43d1-a023-bfd413314879 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:59:10.948107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cff7f-922e-43d1-a023-bfd413314879 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:59:10.951607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b73f6a-f3fd-48f6-abfb-eb3b9bdeb8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:59:10.952240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b73f6a-f3fd-48f6-abfb-eb3b9bdeb8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:10.952549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b73f6a-f3fd-48f6-abfb-eb3b9bdeb8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:10.952825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b73f6a-f3fd-48f6-abfb-eb3b9bdeb8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:10.961599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b73f6a-f3fd-48f6-abfb-eb3b9bdeb8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:59:10.963013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b73f6a-f3fd-48f6-abfb-eb3b9bdeb8f6 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:59:10.972400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 768/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:14.703358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcacdc-27bf-4d07-b54b-92f4a088ee0b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:14.703905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcacdc-27bf-4d07-b54b-92f4a088ee0b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:14.703905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcacdc-27bf-4d07-b54b-92f4a088ee0b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:14.705320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:59:14.710525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15b37-d0fd-4f04-9528-95e166efa74f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:14.710943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15b37-d0fd-4f04-9528-95e166efa74f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:14.711074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15b37-d0fd-4f04-9528-95e166efa74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:14.718015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=81 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:59:14.730812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c1e15b37-d0fd-4f04-9528-95e166efa74f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:59:14.732036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c1e15b37-d0fd-4f04-9528-95e166efa74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:14.739858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:59:15.042617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcacdc-27bf-4d07-b54b-92f4a088ee0b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:59:15.044258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dddcacdc-27bf-4d07-b54b-92f4a088ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:15.113025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:59:15.132909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcacdc-27bf-4d07-b54b-92f4a088ee0b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:59:15.137374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab666-d4a4-4fb4-b49c-7b455c8cd485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:59:15.138053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab666-d4a4-4fb4-b49c-7b455c8cd485 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:15.138333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab666-d4a4-4fb4-b49c-7b455c8cd485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:15.138607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab666-d4a4-4fb4-b49c-7b455c8cd485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:15.148025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab666-d4a4-4fb4-b49c-7b455c8cd485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:59:15.149577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ab666-d4a4-4fb4-b49c-7b455c8cd485 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:59:15.159030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 767/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:59:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:17.198109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d29f-ed3f-491f-8cf2-45f78a476af1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:17.198727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d29f-ed3f-491f-8cf2-45f78a476af1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:17.198727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d29f-ed3f-491f-8cf2-45f78a476af1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:17.200066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 13:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 13:59:17.205393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55160d0c-4b5f-41dd-8c37-313c666f5ccd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:17.205840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55160d0c-4b5f-41dd-8c37-313c666f5ccd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:17.205948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55160d0c-4b5f-41dd-8c37-313c666f5ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:17.224150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-55160d0c-4b5f-41dd-8c37-313c666f5ccd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 13:59:17.225384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-55160d0c-4b5f-41dd-8c37-313c666f5ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:17.235211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 13:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 13:59:17.533556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044d29f-ed3f-491f-8cf2-45f78a476af1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 13:59:17.535232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2044d29f-ed3f-491f-8cf2-45f78a476af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 13:59:17.548946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044d29f-ed3f-491f-8cf2-45f78a476af1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 13:59:17.552430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b90bc-6980-48e4-8f6c-163e04a77534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 13:59:17.553093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b90bc-6980-48e4-8f6c-163e04a77534 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 13:59:17.553371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b90bc-6980-48e4-8f6c-163e04a77534 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 13:59:17.553643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b90bc-6980-48e4-8f6c-163e04a77534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 13:59:17.562595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b90bc-6980-48e4-8f6c-163e04a77534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 13:59:17.564008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b90bc-6980-48e4-8f6c-163e04a77534 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 13:59:17.573373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 771/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 13:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 13:59:27.219084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=11 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 13:59:27.569161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=8 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:48.567712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.568413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.568413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.593487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:48.594725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:48.603597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 769/1540] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:00:48.612970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.613545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.613545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.642535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:48.643765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:48.657196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 772/1541] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:48.662119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:48.675301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36d5a-9206-4ef2-8c6a-65a0cc027844 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:48.681725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:48.682951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.683241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.683510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.684252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:48.684588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:48.686265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:48.702690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 770/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:48.711788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:48.728029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8f4ad-41f0-4e65-a444-eb18a7a4c751 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:48.738245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e124fa-dfb3-4e57-a8eb-e483264f4804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:48.738943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e124fa-dfb3-4e57-a8eb-e483264f4804 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.739232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e124fa-dfb3-4e57-a8eb-e483264f4804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.739507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e124fa-dfb3-4e57-a8eb-e483264f4804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.740245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e124fa-dfb3-4e57-a8eb-e483264f4804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:48.742397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e124fa-dfb3-4e57-a8eb-e483264f4804 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:48.796852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 773/1543] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:48 2022] POST /identity/v3/projects => generated 391 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:00:48.802355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:48.816364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b330df2-5b36-4eeb-8c07-12a0c8cf7c22 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:48.897340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.897690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.897905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.915430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1757b43-641e-4688-942e-52efe9de9467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:48.916317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1757b43-641e-4688-942e-52efe9de9467 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.916642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1757b43-641e-4688-942e-52efe9de9467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.917531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1757b43-641e-4688-942e-52efe9de9467 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.918244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1757b43-641e-4688-942e-52efe9de9467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:48.920814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1757b43-641e-4688-942e-52efe9de9467 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:48.937814 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:48.939122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:48.965934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 771/1544] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:48 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:00:48.967387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750920c0-37e9-475b-af40-53ce589b7880 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:48.967817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750920c0-37e9-475b-af40-53ce589b7880 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:48.968905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 774/1545] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:48.972459 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750920c0-37e9-475b-af40-53ce589b7880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:48.973414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.001013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28cfcd-5e08-4a52-a57e-b34820798155 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.007872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.008295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-750920c0-37e9-475b-af40-53ce589b7880 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.008597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.008881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.009156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.009520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-750920c0-37e9-475b-af40-53ce589b7880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.009901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.012833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.029922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 772/1546] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.030989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14b451-c9cb-4d1d-8fd5-43a8acb9bfce None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.031272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14b451-c9cb-4d1d-8fd5-43a8acb9bfce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.031543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14b451-c9cb-4d1d-8fd5-43a8acb9bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.036271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 775/1547] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:49.040149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.063483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb278bb-c7c5-4ba3-b320-ce8fb2c9aa33 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.066220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0f14b451-c9cb-4d1d-8fd5-43a8acb9bfce None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.078947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95510f-ab14-4f21-b739-5e9e63829e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.079655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95510f-ab14-4f21-b739-5e9e63829e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.079940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95510f-ab14-4f21-b739-5e9e63829e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.080222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95510f-ab14-4f21-b739-5e9e63829e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.081017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95510f-ab14-4f21-b739-5e9e63829e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.082764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95510f-ab14-4f21-b739-5e9e63829e47 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.084911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0f14b451-c9cb-4d1d-8fd5-43a8acb9bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.111865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 773/1549] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.112478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 776/1549] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:49.113260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.113552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.113832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.115851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.116484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.116981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.175030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.175405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.183326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.184325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.205938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 777/1550] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.207812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.209736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 774/1551] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.211403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.230908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d29e-4fcb-4c38-a5d6-b71e415080a0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.233519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda4d1a-ae60-4381-a00c-a6470ca670c4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.242191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.243133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.244309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.245885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.246092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.246343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.246622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.247221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] PATH_INFO: `/v3/projects/77c8c864a31b402880c334c39fbe5a34/users/b86c21e37e2a49e9bdd48e15b091ce09/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.247412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:49.247676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.249398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd39f27-321d-4afa-bf6d-ac44d64d9624 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.283653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 775/1552] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:49.285514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c259-e474-4912-beeb-63143b538d21 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.286074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c259-e474-4912-beeb-63143b538d21 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.286580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c259-e474-4912-beeb-63143b538d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.293922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c8c864a31b402880c334c39fbe5a34, user_id=b86c21e37e2a49e9bdd48e15b091ce09, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.296978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.308640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ab48c259-e474-4912-beeb-63143b538d21 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.310120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab48c259-e474-4912-beeb-63143b538d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.313084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 778/1553] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:49 2022] PUT /identity/v3/projects/77c8c864a31b402880c334c39fbe5a34/users/b86c21e37e2a49e9bdd48e15b091ce09/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:49.313967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.373580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 776/1554] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.374902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c7288-4d6d-4feb-91b4-0cdf7a971dab None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.375197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c7288-4d6d-4feb-91b4-0cdf7a971dab None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.375483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c7288-4d6d-4feb-91b4-0cdf7a971dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.376292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5816af3-96a9-42df-8bcf-e066fad6c6e7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.381292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.382002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.382288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.382567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.383349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:49.383658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.384727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 14:00:49.384727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 14:00:49.384727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 14:00:49.386211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4124-0b7e-4084-b065-db09d960313f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.395065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-091c7288-4d6d-4feb-91b4-0cdf7a971dab None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.396300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-091c7288-4d6d-4feb-91b4-0cdf7a971dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.398790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 779/1555] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:49.400150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.400492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.400804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.412059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 777/1556] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.413537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355790a-4438-4447-82af-f2355e3a6644 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.413844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355790a-4438-4447-82af-f2355e3a6644 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.414130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355790a-4438-4447-82af-f2355e3a6644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.422246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.423477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.431684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 780/1557] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.433051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.438582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8355790a-4438-4447-82af-f2355e3a6644 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.439816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8355790a-4438-4447-82af-f2355e3a6644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.444616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bb887-664f-4862-9d4e-d2e656fd717e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.447954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 778/1558] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.449810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5626a63-352d-4064-acd9-e88cc32f7814 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.450070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.450218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5626a63-352d-4064-acd9-e88cc32f7814 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.450383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5626a63-352d-4064-acd9-e88cc32f7814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.450733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.451017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.451296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.452003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.453972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.468721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e5626a63-352d-4064-acd9-e88cc32f7814 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.469953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e5626a63-352d-4064-acd9-e88cc32f7814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.478594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.481139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.481603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.481888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.487016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 781/1560] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:49.487854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.504665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.506047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.519721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:49.524113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:49.558982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04b1ac-4c4d-4a44-8cfe-a7b3b38ce337 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.563869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.564580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.564866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.565277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.566018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:49.566329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.567941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27426dfa-1981-4a44-90b5-871ef04eeeff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.572434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 780/1561] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.572434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.581569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff1c13-3ba3-4052-a685-707a86a31f4c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.590901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 782/1562] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:49.590901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eac245-ec22-4d5f-85f3-a5933cb9f7a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.590901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eac245-ec22-4d5f-85f3-a5933cb9f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.590901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eac245-ec22-4d5f-85f3-a5933cb9f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.594364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.595091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.595385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.595794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.596583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.597618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 14:00:49.597618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 14:00:49.597618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 14:00:49.598841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f1f81-823a-45bc-abe8-58bb6ec01114 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.606356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e5eac245-ec22-4d5f-85f3-a5933cb9f7a1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.608081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e5eac245-ec22-4d5f-85f3-a5933cb9f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.617388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 783/1563] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.618658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.618947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.619228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.627526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 781/1564] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:49.628071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.628357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.628668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.638471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.639684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.647302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.648531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.711655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 784/1565] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.712610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.724621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 782/1566] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.725637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.736123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43e08-ba2f-4a69-a201-ed2c16056e3c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.741151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01143650-d4fe-4aef-9fd8-af0a70e14467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.741847 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143650-d4fe-4aef-9fd8-af0a70e14467 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.742130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143650-d4fe-4aef-9fd8-af0a70e14467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.742408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143650-d4fe-4aef-9fd8-af0a70e14467 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.743120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01143650-d4fe-4aef-9fd8-af0a70e14467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.745186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01143650-d4fe-4aef-9fd8-af0a70e14467 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.767745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 783/1567] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:49.769491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.769784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.770065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.781219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca0653-a793-4d40-b1fb-55badc23e16f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.785337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.786023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.786298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.786570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.791105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:49.796864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:49.798071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.799659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.808754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 784/1568] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:49.809919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.813330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 785/1569] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:49.814231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.820651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee0540-bc8e-4398-b2ba-2da1fa62b593 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.824747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237fa9f-859f-4fbe-a849-d004c1da0bee service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.825937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.826654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.826941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.827220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.827937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.829988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.830958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.831844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.832309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.832617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.833385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:49.833716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.835466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.850704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 786/1570] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:49.851586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.863253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc65d2-724f-4742-a8ec-6c1b45c093d6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.868572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.872562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 785/1571] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:49.872686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.872686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.872881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.872971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.872971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.872971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 14:00:49.872971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 14:00:49.872971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 14:00:49.873983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.899713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 787/1572] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:49 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:49.899713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:49.909139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596e6ed-7f3b-4215-866e-42a0bbc2f34c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.945746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:49.952311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:49.970504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086904f2-1c18-48a3-afcc-dfdbf9d1f623 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:49.977730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.979493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.979878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.980788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.981804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:49.982361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.984556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:49.985673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:49.986511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:49.986812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:49.987340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:49.988358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:49.989065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:49.990814 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.008028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 786/1573] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:50.008028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.017034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 788/1574] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:50.017904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccf23b-df2f-4737-8f6e-28df7ec25668 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.023237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.037560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.038269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.038565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.039227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.039366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e544b-df83-4698-b369-67cf5fb561da admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.039991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.042910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.045528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.046447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.046872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.047235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.048106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:00:50.048571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.050489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.056456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 787/1575] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.057156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.069833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 789/1576] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:00:50.070229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63e58a-ff6a-46ae-bf9e-a9179fefcc9b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.076331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.077057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.077391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.077713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.078446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.079170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.080485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.092267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f4479-57ad-422a-96dc-ed68086a7065 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.099089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.099791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.100080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.100360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.101104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.103119 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.116785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 788/1577] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.116785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.137323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 790/1578] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/users => generated 287 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.137323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.153493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb0a5f-80b7-434d-a1e4-007e2606acd7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.221916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d56f-0597-4703-8889-962d1a733436 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.232591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.233447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.233819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.234109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.235021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.236901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.238060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.239157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.240114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.240541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.241361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.243877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.252341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 789/1579] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.253285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.271390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb54b-b19f-44ba-be2e-a3cdb8ad6318 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.279793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.280723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.281072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.281353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.282099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.284086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.286264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 791/1580] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.287236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.319546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 790/1581] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.320518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.367953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d4907-f6ad-492b-971b-23f050392625 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.389311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:50.403910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f2ac-8d30-4b21-8ee1-4fb6d6e5dd3a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.409993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.410691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.410982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.411266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.412408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.415307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.431444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.432213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.432668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.433083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.433834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.435762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.446912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 792/1582] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.447670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.447807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 791/1583] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.449169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.463467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:50.465339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476557bb-8caa-4b0c-acde-e97ca1a15a3b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.540921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a1d4e-5885-4ba7-bbec-28f2c464901c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.544649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.545556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.545920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.546675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.547160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.548517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.549282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.549411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.549710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.550097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.551117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.553057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.573787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 792/1584] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.573787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.575178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 793/1585] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.579294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.587131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17da855-437f-4631-ac9c-f4a3e9e1821f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.591111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8c89-2dfa-4287-abcb-f6c9b66cc3e9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.593512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.594729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.595215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.595545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.596943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.597379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.597924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.598316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.598789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.599183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.599697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.601895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.615804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 793/1586] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.616985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.628738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c90f1-3729-4731-9888-1047fcaef4a7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.629621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 794/1587] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/users => generated 288 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.633442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.634146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.634893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.635183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.636150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.637196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.639245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.647017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c58b5-226b-4619-a072-821badcceb2b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.654095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.655837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.656350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.657234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] PATH_INFO: `/v3/projects/c160b9f06dad4ee98349dbc63131c20d/users/520583af1f264f22a86722b0cb9ff405/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.672856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 794/1588] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/users => generated 284 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.674254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.680930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c160b9f06dad4ee98349dbc63131c20d, user_id=520583af1f264f22a86722b0cb9ff405, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.684954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.689035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f23ab-48ae-47a1-9b21-f9c199e2473c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.696462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.697924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.699401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.699894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.700836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.703001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.707865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 795/1589] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:50 2022] PUT /identity/v3/projects/c160b9f06dad4ee98349dbc63131c20d/users/520583af1f264f22a86722b0cb9ff405/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:50.709296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.731580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 795/1590] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/users => generated 294 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:50.732975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.745646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabecf6-6e04-4bc5-8621-3818b5d2d278 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.800033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6f5b-d86c-47b3-b73e-12b1f3d21408 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.805716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.806630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.806925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.807203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.808532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.810709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.826037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 796/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.827491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.835894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:50.838818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.839887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.840237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.840893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.841746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.842682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab2483-9733-40a2-b2c0-13a925b2e430 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.843911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.849934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.851659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.852158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.852820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] PATH_INFO: `/v3/projects/5807489ec0c7444f9f38f0a90393f80c/users/4e9933d1d47c41379aebf2d0208fa5d9/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.862587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 796/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.863785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.879367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5807489ec0c7444f9f38f0a90393f80c, user_id=4e9933d1d47c41379aebf2d0208fa5d9, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.879367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.879782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45343f7b-9653-4fd4-ba38-bed4bdcfe982 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.884936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78a46-40ce-4c39-b042-a5b2dac2a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.886601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78a46-40ce-4c39-b042-a5b2dac2a0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.886898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78a46-40ce-4c39-b042-a5b2dac2a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.887569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78a46-40ce-4c39-b042-a5b2dac2a0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.888326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c78a46-40ce-4c39-b042-a5b2dac2a0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:50.890865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c78a46-40ce-4c39-b042-a5b2dac2a0e4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:50.896608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 797/1593] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:50 2022] PUT /identity/v3/projects/5807489ec0c7444f9f38f0a90393f80c/users/4e9933d1d47c41379aebf2d0208fa5d9/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:50.906147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:50.917984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 797/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:50.919951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.920624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:50.921757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:50.957864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:50.959151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:50.992097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad9c70-b47d-40a4-a891-53243940c895 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:50.998574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:50.999474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:50.999763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.000045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.001478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.002686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.010943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:51.021810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 798/1595] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:51.026699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.037756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 798/1596] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:51.039214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.046729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cab90f-ebb9-46ea-ad5d-4393294f9731 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.054310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.055663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.056106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.056414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.058197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.059825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.059958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a0d45-b740-4aca-821a-7163e83c2718 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.066345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.067535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.067893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.068288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] PATH_INFO: `/v3/projects/a98b3700ace44b199bcf6bbdddb05cca/users/f59649e585784a24bf7addb18e76a18c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.074367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 799/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:51.080837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.092545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98b3700ace44b199bcf6bbdddb05cca, user_id=f59649e585784a24bf7addb18e76a18c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.093746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 14:00:51.093746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 14:00:51.093746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 14:00:51.095999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1761e21-fff8-4f2a-bd40-e98e511a4100 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.101154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ed8e1-b0cb-40c2-9fb1-dc570543217e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.108920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.109859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.110663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.111156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.112306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.114509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.115911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 799/1598] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:51 2022] PUT /identity/v3/projects/a98b3700ace44b199bcf6bbdddb05cca/users/f59649e585784a24bf7addb18e76a18c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:51.117637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.118069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.118583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.142139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 800/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:51.143500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.157993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] MFA Rules not processed for user `4e9933d1d47c41379aebf2d0208fa5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:51.159336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:51.159830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867136e-6154-4859-9c29-25173dcfe616 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.239653 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.240841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.241206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.241564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.242931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.246125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.249257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 800/1600] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:51.263758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 801/1601] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:51.263758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.264072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.264662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:51.279031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f452c-b650-44d6-92a8-aac122269d3e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.284171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.285084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.285420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.285704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.286510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.288209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.301773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 801/1602] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:51.302588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.319160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db580-b7c4-4a19-bbe7-ed9ab06197c2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.327748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24d17-f6ba-486f-9056-4f35d9d6c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.330491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24d17-f6ba-486f-9056-4f35d9d6c98b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.330617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24d17-f6ba-486f-9056-4f35d9d6c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.330938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24d17-f6ba-486f-9056-4f35d9d6c98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.331669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24d17-f6ba-486f-9056-4f35d9d6c98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.335685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24d17-f6ba-486f-9056-4f35d9d6c98b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.352628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8c6e1-95be-4b74-8f41-715cbc5b7758 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.359855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.360584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.361094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 802/1603] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:51.362458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba1479-f76f-4504-bdc7-0868f6a30c94 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.362773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba1479-f76f-4504-bdc7-0868f6a30c94 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.363053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba1479-f76f-4504-bdc7-0868f6a30c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.364135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.364485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.373899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:51.382581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.384047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.410860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 802/1604] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:51.413503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.417205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-59ba1479-f76f-4504-bdc7-0868f6a30c94 None None] MFA Rules not processed for user `f59649e585784a24bf7addb18e76a18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:51.420084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-59ba1479-f76f-4504-bdc7-0868f6a30c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:51.425527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade7456-d245-4c80-9dfb-3fb18f96764a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.434872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.435921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.436201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.436639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] PATH_INFO: `/v3/projects/d3c9ca821cd341a98f21ca15a0d6eb4c/users/13e70b3794a445fc8deeb5f967eef673/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.459712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c9ca821cd341a98f21ca15a0d6eb4c, user_id=13e70b3794a445fc8deeb5f967eef673, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.467216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.493686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 803/1605] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:51 2022] PUT /identity/v3/projects/d3c9ca821cd341a98f21ca15a0d6eb4c/users/13e70b3794a445fc8deeb5f967eef673/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:51.495500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.571458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 803/1606] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:51.573613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcf66a-d6f0-421f-a98b-3241af51f6d4 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.573942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcf66a-d6f0-421f-a98b-3241af51f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.574650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcf66a-d6f0-421f-a98b-3241af51f6d4 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.576332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 804/1607] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jul 15 14:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:00:51.577612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcf66a-d6f0-421f-a98b-3241af51f6d4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.619649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:51.778012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385fcc4-a399-44af-923a-bdf073af8889 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.778012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.779111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcf66a-d6f0-421f-a98b-3241af51f6d4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:51.779745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.779745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.779745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] PATH_INFO: `/v3/projects/c8778d5da3de4c4dbae5d24842b359bc/users/96d4c389c4da4d2188e8a32e5ed1be7d/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.780270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d332e5a-cb0f-43c8-ae66-0ebb5e06b8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:51.780270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d332e5a-cb0f-43c8-ae66-0ebb5e06b8ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.780270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d332e5a-cb0f-43c8-ae66-0ebb5e06b8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.780270 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d332e5a-cb0f-43c8-ae66-0ebb5e06b8ea admin admin] PATH_INFO: `/v3/projects/f6ccc426e62e4a7994832ac941b790d8/users/39e6363a20be44cd9c9feb0c6dbab857/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.780944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8778d5da3de4c4dbae5d24842b359bc, user_id=96d4c389c4da4d2188e8a32e5ed1be7d, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.785414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.808455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d332e5a-cb0f-43c8-ae66-0ebb5e06b8ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ccc426e62e4a7994832ac941b790d8, user_id=39e6363a20be44cd9c9feb0c6dbab857, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:51.825863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d332e5a-cb0f-43c8-ae66-0ebb5e06b8ea admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:51.838062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 804/1608] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:51 2022] PUT /identity/v3/projects/c8778d5da3de4c4dbae5d24842b359bc/users/96d4c389c4da4d2188e8a32e5ed1be7d/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:51.841388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:51.874200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 805/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:51 2022] PUT /identity/v3/projects/f6ccc426e62e4a7994832ac941b790d8/users/39e6363a20be44cd9c9feb0c6dbab857/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:51.874200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:51.874200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:51.874200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:51.897582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:51.898986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:51.948552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355421-d5d6-48ed-b8d8-5219a14b7282 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.010394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 806/1610] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:52.011655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.027362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78b071-a360-4882-aec2-937dcac1ff17 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.033231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff00e0-0883-42fd-85ad-48e36be325fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.034252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff00e0-0883-42fd-85ad-48e36be325fb admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.034546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff00e0-0883-42fd-85ad-48e36be325fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.034957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff00e0-0883-42fd-85ad-48e36be325fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.036013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff00e0-0883-42fd-85ad-48e36be325fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.038641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff00e0-0883-42fd-85ad-48e36be325fb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.075485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f12c59-d55d-4d0b-9022-e2a2e542a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.076549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f12c59-d55d-4d0b-9022-e2a2e542a8f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.078888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f12c59-d55d-4d0b-9022-e2a2e542a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.079277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f12c59-d55d-4d0b-9022-e2a2e542a8f0 admin admin] PATH_INFO: `/v3/projects/5a4b534d33fa4e0a9aca57aad9e74180/users/e81bf1642c95434c85f0cb0816a48c0c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.084325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:52.087405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 807/1611] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:52.097576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.098012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.098471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.108332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f12c59-d55d-4d0b-9022-e2a2e542a8f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4b534d33fa4e0a9aca57aad9e74180, user_id=e81bf1642c95434c85f0cb0816a48c0c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.112891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f12c59-d55d-4d0b-9022-e2a2e542a8f0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.160241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 805/1612] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:51 2022] PUT /identity/v3/projects/5a4b534d33fa4e0a9aca57aad9e74180/users/e81bf1642c95434c85f0cb0816a48c0c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:52.160241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a24ccc-ecd2-451f-be52-01e4e4da9bb7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.160241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a24ccc-ecd2-451f-be52-01e4e4da9bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.160241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a24ccc-ecd2-451f-be52-01e4e4da9bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.185396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-79a24ccc-ecd2-451f-be52-01e4e4da9bb7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:52.188283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79a24ccc-ecd2-451f-be52-01e4e4da9bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:52.199775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:52.204427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:52.286532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 806/1613] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:52.286532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.286532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.286532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.305936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:52.325078 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] MFA Rules not processed for user `39e6363a20be44cd9c9feb0c6dbab857`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:52.326497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:52.336056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 808/1614] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:52.346006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.463454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 807/1615] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:52.464633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.475506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e1318-733f-4d1d-95de-7aa2bb490b09 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.482746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872fa95-7bad-4ed6-922f-08db81403200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.483436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39e6ba-66b3-481d-b25f-4228ae2d3ecd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.493000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872fa95-7bad-4ed6-922f-08db81403200 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.493000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872fa95-7bad-4ed6-922f-08db81403200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.493000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872fa95-7bad-4ed6-922f-08db81403200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.498235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.498942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.499227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.499508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.500229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.502290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.510135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872fa95-7bad-4ed6-922f-08db81403200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.512145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872fa95-7bad-4ed6-922f-08db81403200 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.523608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 809/1616] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:52.537205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521e01d-137c-4c36-a772-98790342a436 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.538755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521e01d-137c-4c36-a772-98790342a436 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.541592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521e01d-137c-4c36-a772-98790342a436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.545333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 808/1617] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/users => generated 292 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:52.546229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.570282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5c3a8-4eed-4aa3-b186-3f5ca493cfc7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.579503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.582710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.583336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.583663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.609616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.612478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.617485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b521e01d-137c-4c36-a772-98790342a436 None None] MFA Rules not processed for user `e81bf1642c95434c85f0cb0816a48c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:52.619683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b521e01d-137c-4c36-a772-98790342a436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:52.648070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 809/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:52.649136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.673259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71864cb0-cccb-42de-88dd-c045aa1adcda service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.686342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.688793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.689301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.689840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.777519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 810/1619] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:52.787905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521e01d-137c-4c36-a772-98790342a436 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.799753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:52.839609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.841325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.864790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 810/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:52.865651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.876598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760141-bc6e-4c4c-aa43-71553aa219c4 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.900828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.902589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.904536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.905182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.907733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:52.911699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:52.927911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521e01d-137c-4c36-a772-98790342a436 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.946693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 811/1621] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:52.949514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:52.957102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.961927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.962236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.962514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:52.975367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e35d37-2e40-4c1f-9674-1fbe6529f850 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:52.991512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:52.995579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:52.996135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:52.996617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.004437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.004437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.006787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.016206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.075466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 811/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:53.076466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.079537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 812/1623] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:52 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:00:53.084275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.136662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf41c4-7d6d-41a9-ab7c-5656ce89c39c service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.238549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:53.248256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b082f-bf2b-4d6e-9c15-2b30e4be6e68 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.265136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.266059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.266445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.266826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.272119 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.274392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.298433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.299690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.300140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.300630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] PATH_INFO: `/v3/projects/1b332a1739824f08bdb8dbbe93b14cad/users/2474f1efb2994a328d3b063de7daa41e/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.330994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 813/1624] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:53 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:00:53.335104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.336869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b332a1739824f08bdb8dbbe93b14cad, user_id=2474f1efb2994a328d3b063de7daa41e, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.340465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.365332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61f119-074a-46eb-9cf9-705e897995ec admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.380448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 812/1625] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:53 2022] PUT /identity/v3/projects/1b332a1739824f08bdb8dbbe93b14cad/users/2474f1efb2994a328d3b063de7daa41e/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:53.386487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.518747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.521758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.522046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.522328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.523114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.532217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.555967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0028310-087e-4321-a245-9068b797f735 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.570285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 814/1626] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:53.570588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.571875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.572180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.575511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.576338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.578017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.586981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.594222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8980aa9-0006-4229-a8b1-58c22c547f57 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.610774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.611054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.612913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.613232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.614065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.619007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.640242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 813/1627] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:53.643008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.650254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 815/1628] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:53.651029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:53.669713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fc1e8-9385-4543-b855-65192e6336c1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.671535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7dc55-3bc5-455f-bb2b-756bb210590c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:53.677906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1b5f8-c491-476c-a239-195b6abee3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.679545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1b5f8-c491-476c-a239-195b6abee3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.680171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1b5f8-c491-476c-a239-195b6abee3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.680825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1b5f8-c491-476c-a239-195b6abee3b5 admin admin] PATH_INFO: `/v3/projects/1b332a1739824f08bdb8dbbe93b14cad/users/2474f1efb2994a328d3b063de7daa41e/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.684266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1b5f8-c491-476c-a239-195b6abee3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b332a1739824f08bdb8dbbe93b14cad, user_id=2474f1efb2994a328d3b063de7daa41e, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.688356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1b5f8-c491-476c-a239-195b6abee3b5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.692203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701625dd-429b-49d6-83df-1c3397381e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:53.699463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701625dd-429b-49d6-83df-1c3397381e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.700144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701625dd-429b-49d6-83df-1c3397381e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.702150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701625dd-429b-49d6-83df-1c3397381e44 admin admin] PATH_INFO: `/v3/projects/73b75a622a23410384f1bd46241b16b7/users/feae6a75be784fa492d54960a27b4c7a/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.709880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 814/1629] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:53 2022] PUT /identity/v3/projects/1b332a1739824f08bdb8dbbe93b14cad/users/2474f1efb2994a328d3b063de7daa41e/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:00:53.711818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763003a-c78f-4fca-902b-3e3cad8304cf None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.712331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763003a-c78f-4fca-902b-3e3cad8304cf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.712921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763003a-c78f-4fca-902b-3e3cad8304cf None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.721983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 815/1630] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jul 15 14:00:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:00:53.724540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d362-5cba-4123-875c-eb8478255b9e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.725322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d362-5cba-4123-875c-eb8478255b9e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.726024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d362-5cba-4123-875c-eb8478255b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.758900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7c86d362-5cba-4123-875c-eb8478255b9e None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:53.760519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7c86d362-5cba-4123-875c-eb8478255b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:53.786580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701625dd-429b-49d6-83df-1c3397381e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b75a622a23410384f1bd46241b16b7, user_id=feae6a75be784fa492d54960a27b4c7a, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:53.790573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701625dd-429b-49d6-83df-1c3397381e44 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:53.819327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:53.840444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 816/1631] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:53 2022] PUT /identity/v3/projects/73b75a622a23410384f1bd46241b16b7/users/feae6a75be784fa492d54960a27b4c7a/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:00:53.840444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:53.840444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:53.840444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:53.877026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] MFA Rules not processed for user `2474f1efb2994a328d3b063de7daa41e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:53.879089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:53.998166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 816/1632] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:00:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:54.000469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.000990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.004813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.057577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 817/1633] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:54.057577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.071266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] MFA Rules not processed for user `feae6a75be784fa492d54960a27b4c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:54.073732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:54.108807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:54.207582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83718b57-8b75-48fe-8628-9781f25d38e0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:54.215569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:54.219498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.219854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.220183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.241030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 817/1634] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:54.245381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.393837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc16d3-5281-43a8-9d06-5c3e90f6dde6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:54.398799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:54.399482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.399767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.400048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.400783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:54.402757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:54.413776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:54.423142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:54.460887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 818/1635] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:54.466582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.470608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 818/1636] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:00:54 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:00:54.474659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.539713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64493b3-69a8-4d30-bc1d-0f6887b443de service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:54.610656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5c00-b29b-46e1-8afd-7ffa7a93b52a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:54.624453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:54.625138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.625423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.625702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.626426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:54.628411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:54.669925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 819/1637] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:00:54 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:00:54.674170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.697419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c55d85-3e30-4211-a146-cd46e4bac0b5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:54.701305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:54.709925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:54.710634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.710923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.711203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.712059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:54.714356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:54.763090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 820/1638] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:54.764361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.769228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:54.771202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.771886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.772339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.780788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505c165-5653-4b72-b01c-57eaae062054 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:54.788427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:54.789684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:54.790187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:54.790588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] PATH_INFO: `/v3/projects/df1af67fc60f46d58204ca3a8816494c/users/b0fb256b4eeb498390008ad3a2e50051/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:54.820786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1af67fc60f46d58204ca3a8816494c, user_id=b0fb256b4eeb498390008ad3a2e50051, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:54.827641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:54.861713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 821/1639] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:54 2022] PUT /identity/v3/projects/df1af67fc60f46d58204ca3a8816494c/users/b0fb256b4eeb498390008ad3a2e50051/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:54.864536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:54.872417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:54.937040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:54.941202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:54.989267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 819/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:55.039237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25516a7-28d0-494d-9da2-d1cbd069ae9c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:55.058953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:55.059661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:55.059952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:55.060228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:55.061077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:55.067972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:55.099580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 822/1641] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:00:55.100283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:55.145444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c0576-52fb-4ad7-9a37-bf0e0300c961 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:55.184072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40a93-fa4b-4d15-a42b-4f5eb6ff34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:55.185393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40a93-fa4b-4d15-a42b-4f5eb6ff34ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:55.190378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40a93-fa4b-4d15-a42b-4f5eb6ff34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:55.190597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40a93-fa4b-4d15-a42b-4f5eb6ff34ae admin admin] PATH_INFO: `/v3/projects/df1af67fc60f46d58204ca3a8816494c/users/b0fb256b4eeb498390008ad3a2e50051/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:55.205439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40a93-fa4b-4d15-a42b-4f5eb6ff34ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1af67fc60f46d58204ca3a8816494c, user_id=b0fb256b4eeb498390008ad3a2e50051, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:55.209021 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40a93-fa4b-4d15-a42b-4f5eb6ff34ae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:55.261245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:55.264340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 820/1642] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:00:55 2022] PUT /identity/v3/projects/df1af67fc60f46d58204ca3a8816494c/users/b0fb256b4eeb498390008ad3a2e50051/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:55.277880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:55.278179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:55.278431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:55.357749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] MFA Rules not processed for user `b0fb256b4eeb498390008ad3a2e50051`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:55.366281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:55.430955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641c07-9718-45e9-9eeb-a7f76ea0c175 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:55.603893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 821/1643] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:55.613363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:55.641602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b689df7-0dbe-45ac-a35b-048f531f5e3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:55.643816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689df7-0dbe-45ac-a35b-048f531f5e3c service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:55.644273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689df7-0dbe-45ac-a35b-048f531f5e3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:55.644731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b689df7-0dbe-45ac-a35b-048f531f5e3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:55.798623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b689df7-0dbe-45ac-a35b-048f531f5e3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:55.800134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b689df7-0dbe-45ac-a35b-048f531f5e3c service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:55.827252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 823/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:00:55.836708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:55.855377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c955596-67e0-406b-a3d5-0c339bfb5521 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:55.877669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fea951-2243-4561-8fb7-4ec48f530c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:55.879319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fea951-2243-4561-8fb7-4ec48f530c39 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:55.879705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fea951-2243-4561-8fb7-4ec48f530c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:55.880232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fea951-2243-4561-8fb7-4ec48f530c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:55.924058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fea951-2243-4561-8fb7-4ec48f530c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:55.935117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fea951-2243-4561-8fb7-4ec48f530c39 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:56.025168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 822/1645] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:56.391112 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c825-24a6-437f-bf44-3407ff71467a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:56.391430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c825-24a6-437f-bf44-3407ff71467a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:56.391716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c825-24a6-437f-bf44-3407ff71467a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:56.393705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 824/1646] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jul 15 14:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:00:56.401603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167e870-22c0-4ddb-98a1-98dc30f44a14 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:56.401893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167e870-22c0-4ddb-98a1-98dc30f44a14 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:56.402177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167e870-22c0-4ddb-98a1-98dc30f44a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:56.438151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5167e870-22c0-4ddb-98a1-98dc30f44a14 None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:56.441147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5167e870-22c0-4ddb-98a1-98dc30f44a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:56.628878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 823/1647] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jul 15 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:56.635773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55785bc-e14f-4b7d-a2ca-e475f88c57ef None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:56.636063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55785bc-e14f-4b7d-a2ca-e475f88c57ef None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:56.636344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55785bc-e14f-4b7d-a2ca-e475f88c57ef None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:56.639169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 825/1648] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jul 15 14:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:00:56.645853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:56.646124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:56.646437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:56.669356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:56.670829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:00:56.697279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 824/1649] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:00:56.786200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55785bc-e14f-4b7d-a2ca-e475f88c57ef None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:56.802885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55785bc-e14f-4b7d-a2ca-e475f88c57ef None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:56.815671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:56.817974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:56.818429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:56.819226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:56.843587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:00:56.849720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:56.851659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:56.904377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 826/1650] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:57.719993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:57.740311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd84af-ec3f-428b-8928-8a65fd6af7fc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:57.758301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5ae59-01ff-4a79-b905-7445042cd359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:57.759006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5ae59-01ff-4a79-b905-7445042cd359 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:57.759294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5ae59-01ff-4a79-b905-7445042cd359 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:57.759573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5ae59-01ff-4a79-b905-7445042cd359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:57.922521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5ae59-01ff-4a79-b905-7445042cd359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:57.925384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5ae59-01ff-4a79-b905-7445042cd359 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:57.949098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 825/1651] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:00:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:00:59.180226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:00:59.426975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc859e-2541-4a4a-93d3-b58fba5c3058 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:00:59.448747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:00:59.451673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:59.451976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:59.452249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] PATH_INFO: `/v3/users/4e9933d1d47c41379aebf2d0208fa5d9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:59.460607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9933d1d47c41379aebf2d0208fa5d9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:00:59.464700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 14:00:59.464700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 15 14:00:59.464700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: Jul 15 14:00:59.472448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:00:59.702673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9933d1d47c41379aebf2d0208fa5d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:00:59.747058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9933d1d47c41379aebf2d0208fa5d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:00:59.761182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9933d1d47c41379aebf2d0208fa5d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:00:59.762195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9933d1d47c41379aebf2d0208fa5d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:00:59.763148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9933d1d47c41379aebf2d0208fa5d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:00:59.806608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 827/1652] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:00:59 2022] DELETE /identity/v3/users/4e9933d1d47c41379aebf2d0208fa5d9 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:00:59.808265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bbb77-a909-49a3-8053-e7a3d0604907 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:00:59.808817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bbb77-a909-49a3-8053-e7a3d0604907 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:00:59.809053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bbb77-a909-49a3-8053-e7a3d0604907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:00:59.857295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8b4bbb77-a909-49a3-8053-e7a3d0604907 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:00:59.859055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8b4bbb77-a909-49a3-8053-e7a3d0604907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:00.084634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 826/1653] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:00.274281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:00.288719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:00.714472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe70ab2-ea5d-49bf-b6b4-7314722e97d6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:00.741324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:00.742418 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:00.742885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:00.743308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:00.831524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:00.833281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:00.864831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 828/1654] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:02.954730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd03c8-afad-4742-80ad-c1bd49129445 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:02.956071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd03c8-afad-4742-80ad-c1bd49129445 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:02.956884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd03c8-afad-4742-80ad-c1bd49129445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.020286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:03.027819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-83fd03c8-afad-4742-80ad-c1bd49129445 None None] MFA Rules not processed for user `b86c21e37e2a49e9bdd48e15b091ce09`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:03.029712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-83fd03c8-afad-4742-80ad-c1bd49129445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.071663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:03.073190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.096123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa77be-c0de-47c9-a895-59cf1376e97a service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:03.114573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:03.115983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:03.117011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:03.117589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] PATH_INFO: `/v3/projects/5807489ec0c7444f9f38f0a90393f80c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.125025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5807489ec0c7444f9f38f0a90393f80c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:03.132676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:03.173940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:03.201349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 827/1655] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:03.213268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:03.213771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:03.214191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.272675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] MFA Rules not processed for user `b86c21e37e2a49e9bdd48e15b091ce09`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:03.284863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.342771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5807489ec0c7444f9f38f0a90393f80c', 'request_id': 'req-390a247b-8111-4cb4-a730-292c346a3b92'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:03.380958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5807489ec0c7444f9f38f0a90393f80c', 'request_id': 'req-390a247b-8111-4cb4-a730-292c346a3b92'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:03.414117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5807489ec0c7444f9f38f0a90393f80c', 'request_id': 'req-390a247b-8111-4cb4-a730-292c346a3b92'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:03.443734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] The token cache is being invalidate because project 5807489ec0c7444f9f38f0a90393f80c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:03.444562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:03.450390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-390a247b-8111-4cb4-a730-292c346a3b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:03.453457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 829/1656] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:03 2022] DELETE /identity/v3/projects/5807489ec0c7444f9f38f0a90393f80c => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:03.464967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f9ff-edd1-44e1-8ef0-d75b705b155f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:03.465707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f9ff-edd1-44e1-8ef0-d75b705b155f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:03.466177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7f9ff-edd1-44e1-8ef0-d75b705b155f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.543964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 828/1657] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:03.546414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-75a7f9ff-edd1-44e1-8ef0-d75b705b155f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:03.548998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-75a7f9ff-edd1-44e1-8ef0-d75b705b155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.557480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:03.558930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.761206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 830/1658] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:03.766867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:03.767315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:03.767608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.808708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7c1c-c1d8-4312-a376-025e90a818a0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:03.833610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:03.834105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:03.834396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:03.834679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.835926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:03.839808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.843262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:03.859656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:03.861695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:03.868810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 831/1659] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:03.873358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:03.891390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119c113-051c-4d18-91ee-dee3ef67c51d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:03.900424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 829/1660] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:03.903577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:03.904697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:03.905151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:03.905442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:03.906890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:03.907206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:03.911355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:03.961784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 832/1661] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:01:03.967399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:03.990043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af121ce-f473-41ce-9706-329500cc1247 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:04.006450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:04.007536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:04.008031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:04.008553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:04.009477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:04.011700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:04.072270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 830/1662] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:03 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:04.076949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:04.098533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964893f-7942-46fa-9007-656b5f7955bf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:04.247492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:04.248213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:04.248545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:04.248832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:04.249554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:04.253801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:04.300874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 833/1663] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:04 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:04.302366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:04.320026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97b5d1-6dae-4c8e-a10f-c51faf160459 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:04.337015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:04.339023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:04.339506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:04.340000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:04.341405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:04.343742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:04.357588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:04.393995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 831/1664] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:04.398581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:04.429136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358c974-a8a2-4085-b194-ee386d7404d2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:04.447091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7333d-a7b8-4f16-9089-8106a76b1b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:04.464983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7333d-a7b8-4f16-9089-8106a76b1b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:04.465451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7333d-a7b8-4f16-9089-8106a76b1b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:04.468448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7333d-a7b8-4f16-9089-8106a76b1b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:04.469413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7333d-a7b8-4f16-9089-8106a76b1b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:04.471208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7333d-a7b8-4f16-9089-8106a76b1b2a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:04.485675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:04.505281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 834/1665] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:04.507129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:04.524683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08142ec-2c88-462a-aac8-908521e09d36 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:04.539244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:04.540511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:04.540943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:04.541366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] PATH_INFO: `/v3/projects/a80d208174a8467b8e3f5941769fb56a/users/0d601cd435f2470094b6baee3f2403d8/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:04.581687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80d208174a8467b8e3f5941769fb56a, user_id=0d601cd435f2470094b6baee3f2403d8, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:04.587648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:04.618217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 832/1666] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:04 2022] PUT /identity/v3/projects/a80d208174a8467b8e3f5941769fb56a/users/0d601cd435f2470094b6baee3f2403d8/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:04.629040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:04.629543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:04.630002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:04.693427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] MFA Rules not processed for user `0d601cd435f2470094b6baee3f2403d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:04.695000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:04.915718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 835/1667] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:04.921649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:05.127884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fac63-cd1d-4b9b-a8da-d04703879e7d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:05.153247 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:05.154063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:05.154499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:05.154918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:05.156922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:05.161204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:05.267001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 833/1668] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:04 2022] POST /identity/v3/projects => generated 363 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:05.268983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:05.306692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f162544-beda-4aa2-8c54-a90faa9713af None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:05.538996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:05.539763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:05.540698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:05.540971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:05.543320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:05.548094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:05.595439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 836/1669] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:05 2022] POST /identity/v3/users => generated 288 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:05.597983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:05.618910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25505fad-8aeb-4642-9350-040efaa10ebc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:05.631373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:05.633664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:05.634318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:05.635016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:05.638290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:05.643876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:05.681665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 834/1670] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:05.683918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:05.724861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a8195-4d3c-48c9-8fbe-585d0be396bf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:05.748174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae78cb-fb51-4372-83de-64117cd1b193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:05.749688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae78cb-fb51-4372-83de-64117cd1b193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:05.750196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae78cb-fb51-4372-83de-64117cd1b193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:05.750566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae78cb-fb51-4372-83de-64117cd1b193 admin admin] PATH_INFO: `/v3/projects/a35a3e1f04ea4e2e9bdea456a58bcdb6/users/16c3baa0a0204ffc83d5dd3b517987fc/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:05.785343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae78cb-fb51-4372-83de-64117cd1b193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35a3e1f04ea4e2e9bdea456a58bcdb6, user_id=16c3baa0a0204ffc83d5dd3b517987fc, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:05.788504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae78cb-fb51-4372-83de-64117cd1b193 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:05.817771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 837/1671] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:05 2022] PUT /identity/v3/projects/a35a3e1f04ea4e2e9bdea456a58bcdb6/users/16c3baa0a0204ffc83d5dd3b517987fc/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:05.819876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:05.858182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdde8cc-292d-4147-bd10-48b0603a07ad None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:05.858480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdde8cc-292d-4147-bd10-48b0603a07ad None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:05.859196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdde8cc-292d-4147-bd10-48b0603a07ad None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:05.860422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 838/1672] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jul 15 14:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:05.869430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:05.870265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:05.870541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:05.878308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7cc96-dd94-4bb7-9b8a-787ce389e8c1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:06.006238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] MFA Rules not processed for user `393c4bea16074727958ea40ab497948d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:06.007528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:06.107337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:06.122565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:06.124055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:06.125365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:06.125796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:06.127003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:06.131422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:06.140680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 839/1673] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:01:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 15 14:01:06.168034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 835/1674] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:06.170142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:06.190430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb249f-10fd-424e-be5e-1792fc311d74 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:06.205526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cba20-dbe0-4a75-aa1c-0f35b3e2aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:06.206445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cba20-dbe0-4a75-aa1c-0f35b3e2aa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:06.206728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cba20-dbe0-4a75-aa1c-0f35b3e2aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:06.207004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cba20-dbe0-4a75-aa1c-0f35b3e2aa41 admin admin] PATH_INFO: `/v3/projects/a35a3e1f04ea4e2e9bdea456a58bcdb6/users/16c3baa0a0204ffc83d5dd3b517987fc/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:06.210497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cba20-dbe0-4a75-aa1c-0f35b3e2aa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35a3e1f04ea4e2e9bdea456a58bcdb6, user_id=16c3baa0a0204ffc83d5dd3b517987fc, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:06.212801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:06.215337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:06.216003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cba20-dbe0-4a75-aa1c-0f35b3e2aa41 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:06.250220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3967d0-5555-488d-840d-5d32411c90ea admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:06.256479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 840/1675] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:06 2022] PUT /identity/v3/projects/a35a3e1f04ea4e2e9bdea456a58bcdb6/users/16c3baa0a0204ffc83d5dd3b517987fc/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:06.261012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd761-dcad-4582-8965-0112330ae417 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:06.262629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd761-dcad-4582-8965-0112330ae417 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:06.262629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288dd761-dcad-4582-8965-0112330ae417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:06.322252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-288dd761-dcad-4582-8965-0112330ae417 None None] MFA Rules not processed for user `16c3baa0a0204ffc83d5dd3b517987fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:06.324024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-288dd761-dcad-4582-8965-0112330ae417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:06.409316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:06.420325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:06.421028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:06.421315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:06.421721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:06.445848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 841/1676] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:06.453691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:06.455455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:06.471629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 836/1677] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:06.586248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dd761-dcad-4582-8965-0112330ae417 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:06.589424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-288dd761-dcad-4582-8965-0112330ae417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:06.742600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288dd761-dcad-4582-8965-0112330ae417 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:06.761852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:06.767236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:06.768962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:06.769407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:06.817517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:06.819650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:06.861017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 842/1678] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:07.084941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:07.108988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61b003-b08b-4a1e-94f5-2ef599b7ee0f service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:07.119936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:07.123166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:07.124474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:07.124934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:07.133818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:07.332756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:07.334604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:07.347148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:07.350323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:07.356776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 837/1679] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:07.502783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:07.504542 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:07.528559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2108417-0e24-4335-b769-d95f9897dd09 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:07.552633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de968985-abc8-49c1-b4a9-988846288246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:07.553409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de968985-abc8-49c1-b4a9-988846288246 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:07.553990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de968985-abc8-49c1-b4a9-988846288246 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:07.553990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de968985-abc8-49c1-b4a9-988846288246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:07.634506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4db80-443f-4c27-8b06-f791ffa7d088 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:07.655003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:07.658040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:07.658783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:07.658895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] PATH_INFO: `/v3/users/39e6363a20be44cd9c9feb0c6dbab857` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:07.667147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e6363a20be44cd9c9feb0c6dbab857)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:07.679070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:07.700160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:07.790169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de968985-abc8-49c1-b4a9-988846288246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:07.795469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de968985-abc8-49c1-b4a9-988846288246 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:07.835492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 838/1680] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:07.837158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e6363a20be44cd9c9feb0c6dbab857'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:07.869360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e6363a20be44cd9c9feb0c6dbab857'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:07.886521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e6363a20be44cd9c9feb0c6dbab857'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:07.887328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e6363a20be44cd9c9feb0c6dbab857'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:07.889266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e6363a20be44cd9c9feb0c6dbab857'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:07.952768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 843/1681] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:07 2022] DELETE /identity/v3/users/39e6363a20be44cd9c9feb0c6dbab857 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:07.958968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34c97e-b2b5-4798-928e-e630a062aa1c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:07.959617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34c97e-b2b5-4798-928e-e630a062aa1c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:07.960096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34c97e-b2b5-4798-928e-e630a062aa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:07.997413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9b34c97e-b2b5-4798-928e-e630a062aa1c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:07.999001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b34c97e-b2b5-4798-928e-e630a062aa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:08.266892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 839/1682] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:08.437742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:08.441760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:08.551511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce6c5-cf21-4732-ace0-467ba225dbb8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:08.552294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce6c5-cf21-4732-ace0-467ba225dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:08.553017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce6c5-cf21-4732-ace0-467ba225dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:08.731632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e91ce6c5-cf21-4732-ace0-467ba225dbb8 None None] MFA Rules not processed for user `520583af1f264f22a86722b0cb9ff405`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:08.734133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e91ce6c5-cf21-4732-ace0-467ba225dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:08.761250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecbf38-ffd2-4378-ba1b-a4fca7c19218 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:08.786836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90fb2f-9cf4-4678-9f01-7b4d1c3d8e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:08.789935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90fb2f-9cf4-4678-9f01-7b4d1c3d8e48 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:08.790588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90fb2f-9cf4-4678-9f01-7b4d1c3d8e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:08.792021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90fb2f-9cf4-4678-9f01-7b4d1c3d8e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:08.792179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:08.850358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90fb2f-9cf4-4678-9f01-7b4d1c3d8e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:08.859368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90fb2f-9cf4-4678-9f01-7b4d1c3d8e48 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:08.919203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 844/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:08.921758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3cc69-c2a6-4a41-8c2d-bc3b1a646aac None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:08.925742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3cc69-c2a6-4a41-8c2d-bc3b1a646aac None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:08.927453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3cc69-c2a6-4a41-8c2d-bc3b1a646aac None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:08.952211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 845/1684] 10.222.0.16 () {54 vars in 960 bytes} [Fri Jul 15 14:01:08 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:08.954538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:08.955226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:08.955962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:09.017962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:09.019644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:09.023622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 840/1685] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:09.029567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:09.029567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:09.029567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:09.093740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] MFA Rules not processed for user `b86c21e37e2a49e9bdd48e15b091ce09`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:09.095010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:09.282816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 841/1686] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:09.284354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:09.288170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:09.316419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 846/1687] 10.222.0.16 () {58 vars in 1057 bytes} [Fri Jul 15 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:09.334097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:09.357918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b1f77-580c-40ce-a3d4-1361de39bfe0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:09.379949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:09.380934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:09.381333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:09.381719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:09.389491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:09.421653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:09.424159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752eea2-852f-4c54-b449-ba03109f4641 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:09.484643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde623c-6019-446b-8440-cd36cfa50d88 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:09.486181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 847/1688] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:09.507683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4e33f-e3c6-4dc8-934a-0c01dfd55a75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:09.508440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4e33f-e3c6-4dc8-934a-0c01dfd55a75 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:09.508781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4e33f-e3c6-4dc8-934a-0c01dfd55a75 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:09.509059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4e33f-e3c6-4dc8-934a-0c01dfd55a75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:09.536018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4e33f-e3c6-4dc8-934a-0c01dfd55a75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:09.537631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4e33f-e3c6-4dc8-934a-0c01dfd55a75 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:09.575593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 842/1689] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:09.671580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53193a-86dc-429b-9b74-f3ad6dfd4821 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:09.672737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53193a-86dc-429b-9b74-f3ad6dfd4821 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:09.673179 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53193a-86dc-429b-9b74-f3ad6dfd4821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:09.753556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:09.807764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9e53193a-86dc-429b-9b74-f3ad6dfd4821 None None] MFA Rules not processed for user `13e70b3794a445fc8deeb5f967eef673`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:09.815874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9e53193a-86dc-429b-9b74-f3ad6dfd4821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:10.110451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 848/1690] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:10.126840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:10.127358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:10.127970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:10.172382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:10.172819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:10.173223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:10.183476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] MFA Rules not processed for user `13e70b3794a445fc8deeb5f967eef673`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:10.185035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:10.210212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 843/1691] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:10.226616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:10.282316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb6ba-cd4e-43ff-b5cd-81c7e069fc86 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:10.299597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] MFA Rules not processed for user `96d4c389c4da4d2188e8a32e5ed1be7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:10.301395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:10.307540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfb074-c075-4f12-b222-9b38961e7630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:10.308328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfb074-c075-4f12-b222-9b38961e7630 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:10.308738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfb074-c075-4f12-b222-9b38961e7630 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:10.309117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfb074-c075-4f12-b222-9b38961e7630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:10.342578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfb074-c075-4f12-b222-9b38961e7630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:10.344308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfb074-c075-4f12-b222-9b38961e7630 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:10.379015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 844/1692] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:10.451030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc687aaf-e145-4325-9982-ef140c0ced43 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:10.451455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc687aaf-e145-4325-9982-ef140c0ced43 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:10.451827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc687aaf-e145-4325-9982-ef140c0ced43 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:10.453475 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 845/1693] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jul 15 14:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 14:01:10.471081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 849/1694] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:10.490369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:10.490894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:10.491318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:10.527051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] MFA Rules not processed for user `393c4bea16074727958ea40ab497948d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:10.530379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:10.575219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 846/1695] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:10.689953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:10.691733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:10.735268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d696165-f10b-4b3a-af06-c63d88aa215d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:10.762956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e803178-d610-42ac-9fee-891aea2d593a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:10.764676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e803178-d610-42ac-9fee-891aea2d593a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:10.764982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e803178-d610-42ac-9fee-891aea2d593a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:10.765247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e803178-d610-42ac-9fee-891aea2d593a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:10.813296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e803178-d610-42ac-9fee-891aea2d593a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:10.817308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e803178-d610-42ac-9fee-891aea2d593a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:10.848149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 850/1696] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:12.129352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:12.131916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:12.168058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97117e6-ad9c-4a9e-9779-62a0d2efd55d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:12.190975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:12.195188 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:12.196836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:12.200879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] PATH_INFO: `/v3/projects/f6ccc426e62e4a7994832ac941b790d8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:12.203709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ccc426e62e4a7994832ac941b790d8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:12.206439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 14:01:12.206439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 15 14:01:12.206439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: Jul 15 14:01:12.209088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:12.283788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:12.408161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ccc426e62e4a7994832ac941b790d8', 'request_id': 'req-ca1ef6a1-2531-4675-ba50-08de02e8f118'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:12.463626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ccc426e62e4a7994832ac941b790d8', 'request_id': 'req-ca1ef6a1-2531-4675-ba50-08de02e8f118'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:12.480306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ccc426e62e4a7994832ac941b790d8', 'request_id': 'req-ca1ef6a1-2531-4675-ba50-08de02e8f118'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:12.531657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] The token cache is being invalidate because project f6ccc426e62e4a7994832ac941b790d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:12.531983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:12.541369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:12.547957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 847/1697] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:12 2022] DELETE /identity/v3/projects/f6ccc426e62e4a7994832ac941b790d8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:12.553670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230868b-3541-4854-b936-afdf068f958c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:12.554167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230868b-3541-4854-b936-afdf068f958c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:12.554436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230868b-3541-4854-b936-afdf068f958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:12.666105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b230868b-3541-4854-b936-afdf068f958c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:12.667393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:12.668994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b230868b-3541-4854-b936-afdf068f958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:12.669124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:12.907361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 851/1698] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:12.908606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230868b-3541-4854-b936-afdf068f958c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:12.910804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b230868b-3541-4854-b936-afdf068f958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:12.998921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef6a1-2531-4675-ba50-08de02e8f118 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:13.021092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:13.022436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:13.022894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:13.023365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] PATH_INFO: `/v3/users/e81bf1642c95434c85f0cb0816a48c0c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:13.029193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81bf1642c95434c85f0cb0816a48c0c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:13.034366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:13.041012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:13.210796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81bf1642c95434c85f0cb0816a48c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:13.281065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81bf1642c95434c85f0cb0816a48c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:13.284431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230868b-3541-4854-b936-afdf068f958c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:13.303835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:13.305374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:13.305374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:13.305541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:13.311621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:13.331769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81bf1642c95434c85f0cb0816a48c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:13.399012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81bf1642c95434c85f0cb0816a48c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:13.399995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-795584d5-fbab-4a2c-b047-fd1b56f4397a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81bf1642c95434c85f0cb0816a48c0c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:13.418141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 848/1699] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:12 2022] DELETE /identity/v3/users/e81bf1642c95434c85f0cb0816a48c0c => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:13.419883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:13.427947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:13.439107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:13.462505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:13.466506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:13.513007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:13.549645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:13.563027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72b987-1a66-4c42-9093-2f5acbdb2f19 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:13.594090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 852/1700] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:13.596446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:13.599107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:13.600767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:13.645918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 849/1701] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:13.655292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:13.663519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:13.666007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:13.704660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861f50b-66a5-4ef0-a8dd-1110a1721daa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:13.716929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 853/1702] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:13.758984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:13.759680 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:13.760637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:13.760926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:13.761733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:13.762057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:13.787392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:13.859880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 850/1703] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:01:13.865079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:13.883544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:13.889022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686f3c2-0c2d-43c1-8452-f4a99f42341a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:13.892815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:13.930834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:13.931454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:13.931741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:13.938618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:13.939434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:13.944802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:14.080794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 854/1704] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:13 2022] POST /identity/v3/projects => generated 405 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:14.085527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:14.209024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e697e9-86ac-499e-be1d-f856526a9fc3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:14.368057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22243403-0122-4de0-9049-a01e5bae6372 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:14.400738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:14.401462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:14.401744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:14.402037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:14.409245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:14.412758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:14.440858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:14.443748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:14.444040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:14.445735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:14.484757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:14.488759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:14.492232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 855/1705] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:14 2022] POST /identity/v3/users => generated 303 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:14.496776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:14.534471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 851/1706] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:14.541148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fc713-d7c0-44d0-89b3-185a57042fcb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:14.568120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:14.568930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:14.569127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:14.569406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:14.570158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:14.579593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:14.598129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 856/1707] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:01:14.601138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:14.613810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae7890-55d4-4dd0-b36f-77d144f44e00 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:14.636907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a148c-e949-4a0b-8bee-58eddf383630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:14.637746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a148c-e949-4a0b-8bee-58eddf383630 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:14.638053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a148c-e949-4a0b-8bee-58eddf383630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:14.638476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a148c-e949-4a0b-8bee-58eddf383630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:14.641245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a148c-e949-4a0b-8bee-58eddf383630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:14.642907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a148c-e949-4a0b-8bee-58eddf383630 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:14.719420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 852/1708] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:14.724267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:14.745459 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:14.758262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b10b-233d-4383-a722-e3024c41f042 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:14.772234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:14.773465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:14.774054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:14.774546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] PATH_INFO: `/v3/projects/f9432998cc514f7ab0240bb051fb9d27/users/d222fb6483e14358a8f35697b369086b/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:14.818046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9432998cc514f7ab0240bb051fb9d27, user_id=d222fb6483e14358a8f35697b369086b, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:14.836817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:14.888009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 857/1709] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:14 2022] PUT /identity/v3/projects/f9432998cc514f7ab0240bb051fb9d27/users/d222fb6483e14358a8f35697b369086b/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:14.893350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:14.893740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:14.894137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:14.966151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:14.967689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:15.194491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:15.221281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 853/1710] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:15.288711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:15.292837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:15.489336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d446d66-3148-4ad4-abd5-e4a2ff947ca3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:15.506444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:15.507138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:15.507726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:15.507726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:15.547722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:15.549245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:15.575064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 858/1711] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:16.129451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:16.131671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:16.167296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2035f-301f-4e24-a8e1-6a1104e8f7a1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:16.175383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:16.177072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:16.177700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:16.178241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] PATH_INFO: `/v3/projects/5a4b534d33fa4e0a9aca57aad9e74180` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:16.180106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4b534d33fa4e0a9aca57aad9e74180)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:16.183286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:16.298095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4b534d33fa4e0a9aca57aad9e74180', 'request_id': 'req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.309640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4b534d33fa4e0a9aca57aad9e74180', 'request_id': 'req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.324183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4b534d33fa4e0a9aca57aad9e74180', 'request_id': 'req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.345509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] The token cache is being invalidate because project 5a4b534d33fa4e0a9aca57aad9e74180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:16.345959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.347486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-730b5d84-5e8c-42d7-b9bd-5e6a0b5845dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.352951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 854/1712] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:16 2022] DELETE /identity/v3/projects/5a4b534d33fa4e0a9aca57aad9e74180 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:16.354606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:16.367589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4041-7880-45c7-9e25-b6d1cd081d36 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:16.367899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4041-7880-45c7-9e25-b6d1cd081d36 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:16.368728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae4041-7880-45c7-9e25-b6d1cd081d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:16.371640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611da406-44c2-474b-b84c-f89239709cad service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:16.415434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-acae4041-7880-45c7-9e25-b6d1cd081d36 None None] MFA Rules not processed for user `520583af1f264f22a86722b0cb9ff405`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:16.421766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-acae4041-7880-45c7-9e25-b6d1cd081d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:16.459235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:16.460557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:16.461033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:16.461462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] PATH_INFO: `/v3/users/feae6a75be784fa492d54960a27b4c7a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:16.464455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feae6a75be784fa492d54960a27b4c7a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:16.466907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:16.575690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 855/1713] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:16.596187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feae6a75be784fa492d54960a27b4c7a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.616821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feae6a75be784fa492d54960a27b4c7a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.633493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feae6a75be784fa492d54960a27b4c7a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.634892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feae6a75be784fa492d54960a27b4c7a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.636295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feae6a75be784fa492d54960a27b4c7a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:16.667711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 859/1714] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:16 2022] DELETE /identity/v3/users/feae6a75be784fa492d54960a27b4c7a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:16.877792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684d742-57f6-4d7b-aa72-3f12c61e2d8c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:16.879200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684d742-57f6-4d7b-aa72-3f12c61e2d8c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:16.879603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684d742-57f6-4d7b-aa72-3f12c61e2d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:16.961587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f684d742-57f6-4d7b-aa72-3f12c61e2d8c None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:16.964194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f684d742-57f6-4d7b-aa72-3f12c61e2d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:17.162580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:17.185464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 856/1715] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:17.373596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:17.375489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:17.385964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:17.567432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c5ef0-a18b-4308-a2da-44f3ea8638f8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:17.618560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae3a0-af4e-43ad-b16f-9d85d9342596 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:17.619857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae3a0-af4e-43ad-b16f-9d85d9342596 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:17.626683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae3a0-af4e-43ad-b16f-9d85d9342596 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:17.629541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae3a0-af4e-43ad-b16f-9d85d9342596 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:17.660181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae3a0-af4e-43ad-b16f-9d85d9342596 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:17.680633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae3a0-af4e-43ad-b16f-9d85d9342596 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:17.729182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 860/1716] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:01:17.939969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:17.940368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:17.940773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:18.005165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] MFA Rules not processed for user `96d4c389c4da4d2188e8a32e5ed1be7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:18.014966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:18.285098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 857/1717] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:18.853386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:18.855119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:18.856006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:18.928672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] MFA Rules not processed for user `d222fb6483e14358a8f35697b369086b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:18.931065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:19.091487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:19.093321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:19.188765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 861/1718] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:19.190068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:19.198463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:19.379981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c68142-f9f9-4b5d-9bf0-5d55cf069459 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:19.398855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:19.430235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:19.431391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:19.439491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] PATH_INFO: `/v3/projects/73b75a622a23410384f1bd46241b16b7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:19.442736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b75a622a23410384f1bd46241b16b7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:19.448712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:19.453170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e782f5-a636-4ca6-89b4-4c9c69082133 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:19.469337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:19.478536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:19.481152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:19.482980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:19.486385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:19.490927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:19.512281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:19.587399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 862/1719] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:19 2022] POST /identity/v3/projects => generated 403 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:19.599861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:19.668422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b75a622a23410384f1bd46241b16b7', 'request_id': 'req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:19.685234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b75a622a23410384f1bd46241b16b7', 'request_id': 'req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:19.721333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b75a622a23410384f1bd46241b16b7', 'request_id': 'req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:19.749966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] The token cache is being invalidate because project 73b75a622a23410384f1bd46241b16b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:19.751674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:19.753852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-282f3d10-52f7-4cb4-8bdc-b36cdf4cb0fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:19.761417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 858/1720] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:19 2022] DELETE /identity/v3/projects/73b75a622a23410384f1bd46241b16b7 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:19.763217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:19.763530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:19.763798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:19.839854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] MFA Rules not processed for user `393c4bea16074727958ea40ab497948d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:19.849777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:19.877438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee749c5-dc2d-46e5-a775-eadb06f8f7bf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:20.009193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:20.099679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 859/1721] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Jul 15 14:01:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:20.101610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:20.122264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:20.131018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:20.132112 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:20.133145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:20.133672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:20.134819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:20.142480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:20.211091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 863/1722] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:19 2022] POST /identity/v3/users => generated 308 bytes in 615 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:20.212546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:20.222919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242f048-02f0-429b-a0b7-c0b9f2d4646f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:20.225614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:20.259931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:20.264812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:20.265101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:20.265775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] PATH_INFO: `/v3/users/b0fb256b4eeb498390008ad3a2e50051` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:20.268684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0fb256b4eeb498390008ad3a2e50051)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:20.274081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:20.354523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4e08a-bad8-4188-a018-8aa38e241acf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:20.382692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:20.386513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:20.394433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:20.394903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:20.466587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0fb256b4eeb498390008ad3a2e50051'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:20.480649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:20.483534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:20.525718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 864/1723] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:20.527051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:20.529039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0fb256b4eeb498390008ad3a2e50051'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:20.538631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:20.584111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0fb256b4eeb498390008ad3a2e50051'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:20.586062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a323f4-07a1-460f-bb8c-3b89d1f7c333 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:20.600318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:20.601583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:20.602011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:20.604222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:20.605131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:20.609821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:20.635800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0fb256b4eeb498390008ad3a2e50051'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:20.637058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0fb256b4eeb498390008ad3a2e50051'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:20.667126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 860/1724] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:20 2022] DELETE /identity/v3/users/b0fb256b4eeb498390008ad3a2e50051 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:20.682374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:20.684120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:20.685951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 865/1725] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:21.040742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc717b-ecaa-4557-9d95-1b1683dbd662 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:21.064116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:21.066802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:21.067094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:21.067425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] PATH_INFO: `/v3/projects/7d269944cb3345c884fca7e673be3177/users/913944a0bc644b7a90532f94b0c303f5/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:21.117569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d269944cb3345c884fca7e673be3177, user_id=913944a0bc644b7a90532f94b0c303f5, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:21.132695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:21.162780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:21.164029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 861/1726] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:20 2022] PUT /identity/v3/projects/7d269944cb3345c884fca7e673be3177/users/913944a0bc644b7a90532f94b0c303f5/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:21.190592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08aac4-72c4-46bf-9287-a2d1c35b265a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:21.323112 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:21.325128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:21.326532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:21.326833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:21.329931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:21.337400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:21.344781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:21.397527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 866/1727] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:21.399217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:21.424318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504bc3f-7657-49a6-b1fb-2b06af6d6d06 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:21.454433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179aa98-cd67-4714-b227-61283c24e44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:21.455673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179aa98-cd67-4714-b227-61283c24e44c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:21.459192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179aa98-cd67-4714-b227-61283c24e44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:21.459630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179aa98-cd67-4714-b227-61283c24e44c admin admin] PATH_INFO: `/v3/projects/7d269944cb3345c884fca7e673be3177/users/913944a0bc644b7a90532f94b0c303f5/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:21.468007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179aa98-cd67-4714-b227-61283c24e44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d269944cb3345c884fca7e673be3177, user_id=913944a0bc644b7a90532f94b0c303f5, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:21.470923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179aa98-cd67-4714-b227-61283c24e44c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:21.500213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 862/1728] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:21 2022] PUT /identity/v3/projects/7d269944cb3345c884fca7e673be3177/users/913944a0bc644b7a90532f94b0c303f5/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:22.273874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:22.277228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:22.308738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f4f5-446c-4e9e-bbad-9b35e0c46378 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:22.467727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:22.469066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:22.469583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:22.470017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] PATH_INFO: `/v3/users/0d601cd435f2470094b6baee3f2403d8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:22.471663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d601cd435f2470094b6baee3f2403d8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:22.475747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:22.657500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d601cd435f2470094b6baee3f2403d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:22.722666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d601cd435f2470094b6baee3f2403d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:22.742622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d601cd435f2470094b6baee3f2403d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:22.743770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d601cd435f2470094b6baee3f2403d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:22.746640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d601cd435f2470094b6baee3f2403d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:22.787949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 867/1729] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:22 2022] DELETE /identity/v3/users/0d601cd435f2470094b6baee3f2403d8 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:22.794424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:22.795097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:22.795579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:22.832644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:22.834569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:22.883265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:23.106523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 863/1730] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:23.132009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:23.134997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:23.372336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb386cf-7c10-42a3-ba97-dbf1b6b844fb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:23.393809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211a97-23a2-4af3-83aa-f1799805f611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:23.396671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211a97-23a2-4af3-83aa-f1799805f611 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:23.397033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211a97-23a2-4af3-83aa-f1799805f611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:23.397385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211a97-23a2-4af3-83aa-f1799805f611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:23.424269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211a97-23a2-4af3-83aa-f1799805f611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:23.432895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211a97-23a2-4af3-83aa-f1799805f611 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:23.504970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 868/1731] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:25.052103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:25.053784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:25.081382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fd301-95ff-4dd5-ad2c-b0853abbf1fc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:25.097639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:25.099106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:25.101807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:25.102391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] PATH_INFO: `/v3/projects/df1af67fc60f46d58204ca3a8816494c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:25.105415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1af67fc60f46d58204ca3a8816494c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:25.112713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:25.245167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1af67fc60f46d58204ca3a8816494c', 'request_id': 'req-c9abea3e-d9db-4610-ac2b-d948c444c38b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:25.270575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1af67fc60f46d58204ca3a8816494c', 'request_id': 'req-c9abea3e-d9db-4610-ac2b-d948c444c38b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:25.287506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1af67fc60f46d58204ca3a8816494c', 'request_id': 'req-c9abea3e-d9db-4610-ac2b-d948c444c38b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:25.304411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] The token cache is being invalidate because project df1af67fc60f46d58204ca3a8816494c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:25.305698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:25.307676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c9abea3e-d9db-4610-ac2b-d948c444c38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:25.318972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:25.319503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:25.319600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:25.321147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 864/1732] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:25 2022] DELETE /identity/v3/projects/df1af67fc60f46d58204ca3a8816494c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:25.350705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:25.353891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:25.409187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:25.576227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:25.577156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:25.577746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:25.580774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 869/1733] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:25.616352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:25.618024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:25.651874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:25.653493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 865/1734] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:25.668080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189643-f552-4be9-ab61-e08e0c350bde None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:25.682935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:25.683793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:25.684182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:25.685260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:25.687712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:25.688123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:25.693179 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:25.731203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 870/1735] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:01:25.732257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:25.748357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153e9e4-6583-4190-97ae-08ebd5cea639 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:25.763239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:25.764318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:25.764794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:25.765225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:25.766222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:25.769564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:25.832608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 866/1736] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:25 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:25.839207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:25.873315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdb30c-6a12-4a2e-b7bd-96cc45735ec1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:26.050517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:26.050517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.052057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.052057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.052057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:26.054709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:26.106716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 871/1737] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:25 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:26.109933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:26.175272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b16003-30d3-4ded-98a6-d5b4900619c5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:26.188915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:26.190498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.190498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.190698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.192063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:26.193766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:26.224521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 867/1738] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:26.229365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:26.243937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bda60-8b17-421b-9b92-6527584cd8a9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:26.250605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:26.251674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.252588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.252935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.253931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:26.256075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:26.275604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 872/1739] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:01:26.279742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:26.295417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:26.299529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:26.303692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f480f8-dd31-444f-a8b2-55cf79b8dcf2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:26.328734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7f776-6dc6-4f44-a44c-18e9021c1b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:26.329667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f776-6dc6-4f44-a44c-18e9021c1b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.332077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f776-6dc6-4f44-a44c-18e9021c1b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.332381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f776-6dc6-4f44-a44c-18e9021c1b12 admin admin] PATH_INFO: `/v3/projects/b6fc3aef706540c8aa2933c3533d4cba/users/41472be82def42b3a3ea9c1b7e49e24c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.363907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea906506-097a-45d7-a25f-4d53020be9c5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:26.381491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:26.382534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.382826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.383103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] PATH_INFO: `/v3/projects/a80d208174a8467b8e3f5941769fb56a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.385799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80d208174a8467b8e3f5941769fb56a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:26.389751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:26.398541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f776-6dc6-4f44-a44c-18e9021c1b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fc3aef706540c8aa2933c3533d4cba, user_id=41472be82def42b3a3ea9c1b7e49e24c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:26.407085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f776-6dc6-4f44-a44c-18e9021c1b12 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:26.457697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 868/1740] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:26 2022] PUT /identity/v3/projects/b6fc3aef706540c8aa2933c3533d4cba/users/41472be82def42b3a3ea9c1b7e49e24c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:26.478534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.481719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.482637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.546246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] MFA Rules not processed for user `41472be82def42b3a3ea9c1b7e49e24c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:26.547608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:26.549766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80d208174a8467b8e3f5941769fb56a', 'request_id': 'req-89070f87-c8db-4c20-bb5a-02909e6ad346'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:26.560202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80d208174a8467b8e3f5941769fb56a', 'request_id': 'req-89070f87-c8db-4c20-bb5a-02909e6ad346'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:26.581011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80d208174a8467b8e3f5941769fb56a', 'request_id': 'req-89070f87-c8db-4c20-bb5a-02909e6ad346'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:26.609511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] The token cache is being invalidate because project a80d208174a8467b8e3f5941769fb56a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:26.609806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:26.616067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:26.617900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 873/1741] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:26 2022] DELETE /identity/v3/projects/a80d208174a8467b8e3f5941769fb56a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:26.622441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:26.624169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:26.639048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:26.662351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:26.805632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 869/1742] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:26.810390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:26.812421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:26.879497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070f87-c8db-4c20-bb5a-02909e6ad346 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:26.901991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:26.902939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:26.903253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:26.903511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] PATH_INFO: `/v3/users/16c3baa0a0204ffc83d5dd3b517987fc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:26.911446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c3baa0a0204ffc83d5dd3b517987fc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:26.913667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:26.994133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c852df4-7c39-4ee3-8703-dee086d82b14 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:27.009523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:27.010256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.010552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.010832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.011555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:27.013594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:27.064439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c3baa0a0204ffc83d5dd3b517987fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:27.080441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 870/1743] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:26 2022] POST /identity/v3/projects => generated 365 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:27.082362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:27.099093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c3baa0a0204ffc83d5dd3b517987fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:27.124298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c3baa0a0204ffc83d5dd3b517987fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:27.126187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c3baa0a0204ffc83d5dd3b517987fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:27.127855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c3baa0a0204ffc83d5dd3b517987fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:27.228153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0475e88-b6c7-4f50-9a97-ac8fba18f4f4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:27.237667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 874/1744] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:26 2022] DELETE /identity/v3/users/16c3baa0a0204ffc83d5dd3b517987fc => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:27.321017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:27.321623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.321909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.322239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.322936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:27.324979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:27.335636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:27.359730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 871/1745] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:27 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:27.363629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:27.365762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:27.403324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86175f-314d-4476-aa5f-d64cd8c77a0b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:27.431081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:27.432374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.438576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.439316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.442078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:27.453968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:27.495735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:27.505424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 875/1746] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:27.525297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8b80b-7ac7-475a-99ea-049277b1188a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:27.530859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:27.531800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.532089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.532375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] PATH_INFO: `/v3/projects/38e234b58f424f55b0bc69bf985bd475/users/72bec2c71fed483496067dc790ac4740/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.552318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e234b58f424f55b0bc69bf985bd475, user_id=72bec2c71fed483496067dc790ac4740, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:27.555672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:27.585644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 872/1747] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:27 2022] PUT /identity/v3/projects/38e234b58f424f55b0bc69bf985bd475/users/72bec2c71fed483496067dc790ac4740/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:27.587013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:27.614906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f303af-ec14-4662-b842-a1f676bde183 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:27.800007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225a24f-ed58-4977-bb15-9b1ca585e23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:27.800921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225a24f-ed58-4977-bb15-9b1ca585e23c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.801389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225a24f-ed58-4977-bb15-9b1ca585e23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.803504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225a24f-ed58-4977-bb15-9b1ca585e23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.804643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225a24f-ed58-4977-bb15-9b1ca585e23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:27.806430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225a24f-ed58-4977-bb15-9b1ca585e23c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:27.825970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 876/1748] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:27.831183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:27.848038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cf2f6-8a44-4090-a006-2ceb9896a684 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:27.878970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b3e01-c439-49d7-bc06-3449ea54f35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:27.881159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b3e01-c439-49d7-bc06-3449ea54f35b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.887671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b3e01-c439-49d7-bc06-3449ea54f35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.889995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b3e01-c439-49d7-bc06-3449ea54f35b admin admin] PATH_INFO: `/v3/projects/38e234b58f424f55b0bc69bf985bd475/users/72bec2c71fed483496067dc790ac4740/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.925278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178d96-a50f-48f2-aea0-5ba15eb1ca66 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.932036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178d96-a50f-48f2-aea0-5ba15eb1ca66 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.932036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178d96-a50f-48f2-aea0-5ba15eb1ca66 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:27.932036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 877/1749] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:27.932514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b3e01-c439-49d7-bc06-3449ea54f35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e234b58f424f55b0bc69bf985bd475, user_id=72bec2c71fed483496067dc790ac4740, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:27.932514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b3e01-c439-49d7-bc06-3449ea54f35b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:27.970860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:27.971551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:27.981116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:28.024882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 873/1750] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:27 2022] PUT /identity/v3/projects/38e234b58f424f55b0bc69bf985bd475/users/72bec2c71fed483496067dc790ac4740/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:28.037431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:28.037753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:28.037999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:28.053133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:28.054383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:28.093827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:28.161410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] MFA Rules not processed for user `72bec2c71fed483496067dc790ac4740`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:28.163677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:28.312223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 878/1751] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:28.348823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 874/1752] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:28.361910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:28.363600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:28.540882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed25c1-b53e-47d8-bc12-926f8ae34316 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:28.552508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:28.553690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:28.554201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:28.554843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:28.582000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:28.586613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:28.611990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 879/1753] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:28.769397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:28.791004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32c314-ab68-48e6-aefe-e94ede3fc451 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:28.795946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:28.797883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:28.798322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:28.798712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:28.801285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:28.932752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:28.934417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64b56d-8ac9-4775-8683-976be836e910 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:28.955715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 875/1754] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:29.446756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:29.448770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:29.499584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a614-3def-407f-9c2a-2d93936f2491 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:29.511954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4741a20-5649-43ef-b869-b2672e7d9e32 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:29.512738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4741a20-5649-43ef-b869-b2672e7d9e32 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:29.512738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4741a20-5649-43ef-b869-b2672e7d9e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:29.521255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:29.524861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:29.525445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:29.526585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:29.543343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:29.546679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:29.565925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 880/1755] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:29.582689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d4741a20-5649-43ef-b869-b2672e7d9e32 None None] MFA Rules not processed for user `913944a0bc644b7a90532f94b0c303f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:29.583944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d4741a20-5649-43ef-b869-b2672e7d9e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:29.652072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:29.732981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 876/1756] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:30.350783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:30.354098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:30.508095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f263de-ee02-49fb-9db9-1652e3a16018 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.509605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f263de-ee02-49fb-9db9-1652e3a16018 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.510139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f263de-ee02-49fb-9db9-1652e3a16018 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.513221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 877/1757] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:01:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:30.527702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.528287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.528797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.549535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c333a6-3ea0-42f9-a551-1639fab41f8a service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:30.562173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:30.563824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:30.565027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.565539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.566028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] PATH_INFO: `/v3/projects/a35a3e1f04ea4e2e9bdea456a58bcdb6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.570693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:30.573100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35a3e1f04ea4e2e9bdea456a58bcdb6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:30.574903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:30.602922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 878/1758] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:30.623498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:30.625488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:30.651082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b2565-57f5-4798-b271-7ed019c682bf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:30.665162 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2812-97d2-46c4-93f3-e1bdf1001c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:30.667755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2812-97d2-46c4-93f3-e1bdf1001c18 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.674236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2812-97d2-46c4-93f3-e1bdf1001c18 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.675558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2812-97d2-46c4-93f3-e1bdf1001c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.687786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:30.699695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2812-97d2-46c4-93f3-e1bdf1001c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:30.701447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2812-97d2-46c4-93f3-e1bdf1001c18 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:30.710393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35a3e1f04ea4e2e9bdea456a58bcdb6', 'request_id': 'req-d10025a0-bd7c-44cf-a302-93326d926203'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:30.727782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35a3e1f04ea4e2e9bdea456a58bcdb6', 'request_id': 'req-d10025a0-bd7c-44cf-a302-93326d926203'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:30.737614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 879/1759] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:30.756499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35a3e1f04ea4e2e9bdea456a58bcdb6', 'request_id': 'req-d10025a0-bd7c-44cf-a302-93326d926203'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:30.774649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] The token cache is being invalidate because project a35a3e1f04ea4e2e9bdea456a58bcdb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:30.780580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:30.781356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d10025a0-bd7c-44cf-a302-93326d926203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:30.795792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 881/1760] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:30 2022] DELETE /identity/v3/projects/a35a3e1f04ea4e2e9bdea456a58bcdb6 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:30.797743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec53c-924c-4eaa-9aa7-19f7f686d557 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.798037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec53c-924c-4eaa-9aa7-19f7f686d557 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.798318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec53c-924c-4eaa-9aa7-19f7f686d557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.849002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6edec53c-924c-4eaa-9aa7-19f7f686d557 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:30.851924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6edec53c-924c-4eaa-9aa7-19f7f686d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:30.927775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b27e33-0a5a-42d1-a682-abfa119b01cf None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.928367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b27e33-0a5a-42d1-a682-abfa119b01cf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.928829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b27e33-0a5a-42d1-a682-abfa119b01cf None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.930486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 882/1761] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:30.938175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:30.938804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:30.939010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:30.978885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:30.982199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:31.050893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 880/1762] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:31.053201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.053599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.054039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.103238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:31.104534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:31.140642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 881/1763] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:31.145077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:31.160806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f2790-c181-413f-8883-b24e80d65c00 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:31.178775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:31.179630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.180088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.180436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.181273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:31.182109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:31.183967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:31.206599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 883/1764] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:31.225098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 882/1765] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:01:31.229632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:31.256381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd6418-ec53-47de-b195-8ac8b0f87a3a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:31.274690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:31.277754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.279549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.280148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.282401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:31.291628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:31.434361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 884/1766] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:31 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:31.435377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:31.457213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16efff-ad04-4547-a8c1-b68390b28d91 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:31.522434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:31.524217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:31.685282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:31.686628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.688522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.688979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.689900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:31.694597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:31.775030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 883/1767] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:31 2022] POST /identity/v3/users => generated 288 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:31.782257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:31.798947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a909fd-b5f7-47e1-9dd4-3bf04d2a8e17 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:31.823101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11290237-4d1b-4911-b8f4-34e589e6f890 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:31.835091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:31.835844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.836133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.836468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.839248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:31.839948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.840419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.844706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.845544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:31.847197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:31.894634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 884/1768] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:31.895630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:31.899884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:31.907063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:31.941360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7ec2-77ca-4697-9140-e5a1759c137c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:31.959408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 885/1769] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:31.977323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c267ea4-d937-4c6a-851b-0ed6bad09b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:31.978049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267ea4-d937-4c6a-851b-0ed6bad09b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:31.978345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267ea4-d937-4c6a-851b-0ed6bad09b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:31.978655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267ea4-d937-4c6a-851b-0ed6bad09b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:31.979420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267ea4-d937-4c6a-851b-0ed6bad09b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:31.996835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c267ea4-d937-4c6a-851b-0ed6bad09b0b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:32.004734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:32.051883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 885/1770] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:32.060452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:32.076685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd26a4-0505-4142-bb2d-b17d755b5541 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:32.095201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:32.096168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:32.096487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:32.096794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] PATH_INFO: `/v3/projects/356871afb8a742258c2128579e7416e5/users/d7af34f911ed4eb0b7b7bd507b392ac8/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:32.174591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356871afb8a742258c2128579e7416e5, user_id=d7af34f911ed4eb0b7b7bd507b392ac8, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:32.177351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:32.231935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 886/1771] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:32 2022] PUT /identity/v3/projects/356871afb8a742258c2128579e7416e5/users/d7af34f911ed4eb0b7b7bd507b392ac8/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:32.233312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:32.233893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:32.234358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:32.359880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] MFA Rules not processed for user `d7af34f911ed4eb0b7b7bd507b392ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:32.371003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:32.702788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 886/1772] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:32.707973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:32.714822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:32.908874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ed914-27d9-4394-9a10-4561f3672524 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:32.922701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:32.923642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:32.924250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:32.924707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:32.925615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:32.927711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:33.021720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 887/1773] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:32 2022] POST /identity/v3/projects => generated 375 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:33.023348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:33.064507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa0f35-6de0-4d0f-9368-f2b9a8d97679 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:33.279952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:33.280714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:33.281080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:33.281434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:33.282227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:33.284477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:33.357863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 887/1774] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:33 2022] POST /identity/v3/users => generated 294 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:33.364331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:33.389663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ca50a-bf2f-4681-9da4-80898df8ee0c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:33.408296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:33.409261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:33.409696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:33.413843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:33.414840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:33.428990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:33.457346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 888/1775] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:33.458399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:33.503497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b3adc-e688-4d34-81ec-29c925db2b74 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:33.537117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:33.538500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:33.539198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:33.539670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] PATH_INFO: `/v3/projects/fabbfac1b93f45a48ca3f5c5ddc4a55a/users/e3d527da08144ca3b11a1114df8b7c2b/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:33.629520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabbfac1b93f45a48ca3f5c5ddc4a55a, user_id=e3d527da08144ca3b11a1114df8b7c2b, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:33.634446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:33.669761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 888/1776] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:33 2022] PUT /identity/v3/projects/fabbfac1b93f45a48ca3f5c5ddc4a55a/users/e3d527da08144ca3b11a1114df8b7c2b/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:33.672723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:33.687672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e496bf-99fa-4ec0-8893-ab66985b8212 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:33.819431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cec52-63b3-434a-b4ed-bf09b369859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:33.820115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cec52-63b3-434a-b4ed-bf09b369859b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:33.820502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cec52-63b3-434a-b4ed-bf09b369859b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:33.820709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cec52-63b3-434a-b4ed-bf09b369859b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:33.821802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cec52-63b3-434a-b4ed-bf09b369859b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:33.824556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cec52-63b3-434a-b4ed-bf09b369859b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:33.845536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 889/1777] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:33.847753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:33.862622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc43be-dd85-43d0-9224-54d71109e7cf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:33.868761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:33.870722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:33.871200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:33.871843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] PATH_INFO: `/v3/projects/fabbfac1b93f45a48ca3f5c5ddc4a55a/users/e3d527da08144ca3b11a1114df8b7c2b/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:33.875207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabbfac1b93f45a48ca3f5c5ddc4a55a, user_id=e3d527da08144ca3b11a1114df8b7c2b, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:33.879027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:33.916728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 889/1778] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:33 2022] PUT /identity/v3/projects/fabbfac1b93f45a48ca3f5c5ddc4a55a/users/e3d527da08144ca3b11a1114df8b7c2b/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:33.917536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb0992-4cba-44ec-a2e2-c6d737769cb9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:33.918030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb0992-4cba-44ec-a2e2-c6d737769cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:33.918236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb0992-4cba-44ec-a2e2-c6d737769cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:33.986366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-19eb0992-4cba-44ec-a2e2-c6d737769cb9 None None] MFA Rules not processed for user `e3d527da08144ca3b11a1114df8b7c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:33.989949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-19eb0992-4cba-44ec-a2e2-c6d737769cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:34.178807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:34.207007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:34.209133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:34.210337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 890/1779] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:34.340004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:34.378112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca76d27-4716-4883-9a0c-921b8443bb81 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:34.384173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b8848-f3d5-410f-853b-560e5769b3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:34.386954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b8848-f3d5-410f-853b-560e5769b3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:34.387412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b8848-f3d5-410f-853b-560e5769b3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:34.387980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b8848-f3d5-410f-853b-560e5769b3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:34.526416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b8848-f3d5-410f-853b-560e5769b3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:34.535985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b8848-f3d5-410f-853b-560e5769b3a6 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:34.568892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 890/1780] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:35.113733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7130d-521d-4cb6-93c5-90a9a576da4e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:35.115305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7130d-521d-4cb6-93c5-90a9a576da4e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:35.115920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7130d-521d-4cb6-93c5-90a9a576da4e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:35.118420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 891/1781] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:01:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:35.238053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:35.240868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:35.241170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:35.308695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] MFA Rules not processed for user `92ac8ddf274a45d1b56533944a2334fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:35.316710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:35.646124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 891/1782] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:35.864161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7130d-521d-4cb6-93c5-90a9a576da4e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:35.891857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:35.893722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:35.912824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7130d-521d-4cb6-93c5-90a9a576da4e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:35.932767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:35.934056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:35.934797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:35.935371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:35.965425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:35.967726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:35.986578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 892/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:36.072760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7936096-e516-47f1-9884-ae551e562e46 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:36.078945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:36.094720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:36.094720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:36.094965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] PATH_INFO: `/v3/users/d222fb6483e14358a8f35697b369086b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:36.106975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] RBAC: Authorizing `identity:delete_user(user_id=d222fb6483e14358a8f35697b369086b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:36.125061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:36.217229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:36.243643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd222fb6483e14358a8f35697b369086b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.265809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd222fb6483e14358a8f35697b369086b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.285729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd222fb6483e14358a8f35697b369086b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.298239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd222fb6483e14358a8f35697b369086b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.299470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd222fb6483e14358a8f35697b369086b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.315548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 892/1784] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:35 2022] DELETE /identity/v3/users/d222fb6483e14358a8f35697b369086b => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:36.762920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:36.787746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70256552-fb96-43cd-98af-95d3e5bdfc48 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:36.860681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:36.861566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:36.862017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:36.862443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] PATH_INFO: `/v3/projects/f9432998cc514f7ab0240bb051fb9d27` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:36.865447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9432998cc514f7ab0240bb051fb9d27)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:36.867608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:36.930114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9432998cc514f7ab0240bb051fb9d27', 'request_id': 'req-27b04b72-8ecf-412d-929c-7e217f7842f7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.939894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9432998cc514f7ab0240bb051fb9d27', 'request_id': 'req-27b04b72-8ecf-412d-929c-7e217f7842f7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.950238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9432998cc514f7ab0240bb051fb9d27', 'request_id': 'req-27b04b72-8ecf-412d-929c-7e217f7842f7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.972892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] The token cache is being invalidate because project f9432998cc514f7ab0240bb051fb9d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:36.973186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.973924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:36.979974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 893/1785] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:36 2022] DELETE /identity/v3/projects/f9432998cc514f7ab0240bb051fb9d27 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:36.981344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:36.984366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:37.004426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79471837-753e-4c1a-b23c-f4fa070ee5af admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:37.076555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:37.077702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.078051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.078515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] PATH_INFO: `/v3/users/913944a0bc644b7a90532f94b0c303f5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.079911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=913944a0bc644b7a90532f94b0c303f5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:37.081964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:37.158239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913944a0bc644b7a90532f94b0c303f5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.176692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913944a0bc644b7a90532f94b0c303f5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.187622 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913944a0bc644b7a90532f94b0c303f5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.198615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913944a0bc644b7a90532f94b0c303f5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.200527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eb079652-dc25-40b0-9055-95d415802e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913944a0bc644b7a90532f94b0c303f5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.213287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 893/1786] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:36 2022] DELETE /identity/v3/users/913944a0bc644b7a90532f94b0c303f5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:37.437508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:37.439320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:37.460778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b04b72-8ecf-412d-929c-7e217f7842f7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:37.541799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:37.542576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.542801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.543083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] PATH_INFO: `/v3/projects/7d269944cb3345c884fca7e673be3177` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.544361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d269944cb3345c884fca7e673be3177)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:37.546334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:37.572165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:37.615404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d269944cb3345c884fca7e673be3177', 'request_id': 'req-94e7a1bd-ac38-4705-9219-f92ee8a943cd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.627067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d269944cb3345c884fca7e673be3177', 'request_id': 'req-94e7a1bd-ac38-4705-9219-f92ee8a943cd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.647608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d269944cb3345c884fca7e673be3177', 'request_id': 'req-94e7a1bd-ac38-4705-9219-f92ee8a943cd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.662982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] The token cache is being invalidate because project 7d269944cb3345c884fca7e673be3177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:37.663284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.663940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.666881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 894/1787] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:37 2022] DELETE /identity/v3/projects/7d269944cb3345c884fca7e673be3177 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:37.672356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf349962-d303-431f-a7ca-8d467ae27400 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.672981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf349962-d303-431f-a7ca-8d467ae27400 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.672981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf349962-d303-431f-a7ca-8d467ae27400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.701064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf349962-d303-431f-a7ca-8d467ae27400 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:37.702459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf349962-d303-431f-a7ca-8d467ae27400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:37.707839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:37.709692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:37.744852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:37.824430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 894/1788] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:37.828001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.828287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.828623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.829567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7a1bd-ac38-4705-9219-f92ee8a943cd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:37.834974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:37.835885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.836171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.836492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] PATH_INFO: `/v3/users/41472be82def42b3a3ea9c1b7e49e24c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.838412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=41472be82def42b3a3ea9c1b7e49e24c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:37.840592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:37.850649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:37.852094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:37.861610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 895/1789] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:37.865842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:37.876985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d68b-7d43-4860-98c8-2dd72d969d29 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:37.882835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:37.883549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.883786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.884066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.885524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:37.885798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:37.887491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:37.903273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 896/1790] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:01:37.904130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41472be82def42b3a3ea9c1b7e49e24c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.909386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:37.918305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41472be82def42b3a3ea9c1b7e49e24c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.920035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9f82f-fe50-4bb6-9e3a-a00ab32668db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:37.926721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:37.926951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41472be82def42b3a3ea9c1b7e49e24c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.927398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.927681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.927960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.928119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41472be82def42b3a3ea9c1b7e49e24c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.928695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:37.929088 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6aebf703-49af-41bd-98ee-eb37fd1e69ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41472be82def42b3a3ea9c1b7e49e24c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:37.930621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:37.949171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 895/1791] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:37 2022] DELETE /identity/v3/users/41472be82def42b3a3ea9c1b7e49e24c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:37.955158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:37.955581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:37.955945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:37.966406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 897/1792] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:37 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:37.967866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:37.970486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:37.983349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:37.985119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:38.009199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:38.049025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7610a-42e2-4cb5-acc1-ebf22871c1cb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.054446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.055141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.055564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.055845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.057167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.059136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.062622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 896/1793] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:38.073717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.075171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:38.083711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 898/1794] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:37 2022] POST /identity/v3/users => generated 279 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:38.090574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.100867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cca149-20f4-477e-9e8b-3e8be1119c95 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.112849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.114102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.121555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.121555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.121555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.121555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.142096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 899/1795] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:38.149277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.173312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53312fcb-a770-4372-b42f-46d1f105a7eb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.174834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca2239-a006-4e42-8889-3bd413b2cc92 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.177423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb03e6-edc5-4e11-b21f-3317a80f8523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.178226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb03e6-edc5-4e11-b21f-3317a80f8523 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.178541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb03e6-edc5-4e11-b21f-3317a80f8523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.178826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb03e6-edc5-4e11-b21f-3317a80f8523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.186530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.187890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.188878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.189540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.191266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.194198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.200179 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb03e6-edc5-4e11-b21f-3317a80f8523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.201852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb03e6-edc5-4e11-b21f-3317a80f8523 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.215682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 900/1796] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:38.219822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 897/1797] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:38.221473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.237178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf667829-d1f7-466e-b05d-4e0e2ea37e38 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.246409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.247605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.248520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.249280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] PATH_INFO: `/v3/projects/759f825197ce4ad29d4476f8cdcffe2d/users/a0ddd975d874424cb50bdfcc13ac1289/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.269521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=759f825197ce4ad29d4476f8cdcffe2d, user_id=a0ddd975d874424cb50bdfcc13ac1289, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.272329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.289531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 901/1798] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:38 2022] PUT /identity/v3/projects/759f825197ce4ad29d4476f8cdcffe2d/users/a0ddd975d874424cb50bdfcc13ac1289/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:38.311841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.312260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.312779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.402655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] MFA Rules not processed for user `a0ddd975d874424cb50bdfcc13ac1289`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:38.404171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:38.514460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 898/1799] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:38.527089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.598239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.600066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:38.620576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:38.674351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b84553-0ff5-479e-a607-5468b9ec5f8f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.682494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.684625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.685332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.685985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.687981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.693191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.701192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:38.706149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4ff2-8052-4684-bf8f-4d3c4bc164f9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.712378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.713924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.714279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.714627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] PATH_INFO: `/v3/projects/b6fc3aef706540c8aa2933c3533d4cba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.715967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fc3aef706540c8aa2933c3533d4cba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.718020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.740440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 902/1800] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:38 2022] POST /identity/v3/projects => generated 357 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:38.743150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.789018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fc3aef706540c8aa2933c3533d4cba', 'request_id': 'req-1da9d673-20f8-4a21-a2a1-0b642fd2f369'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:38.796558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fc3aef706540c8aa2933c3533d4cba', 'request_id': 'req-1da9d673-20f8-4a21-a2a1-0b642fd2f369'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:38.808478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fc3aef706540c8aa2933c3533d4cba', 'request_id': 'req-1da9d673-20f8-4a21-a2a1-0b642fd2f369'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:38.850001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] The token cache is being invalidate because project b6fc3aef706540c8aa2933c3533d4cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:38.850662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:38.852821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:38.857007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 899/1801] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:38 2022] DELETE /identity/v3/projects/b6fc3aef706540c8aa2933c3533d4cba => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:38.863268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49ec1-1477-4485-8554-e08aee08d66f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:38.864588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.866045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:38.942507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:38.943660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:38.944109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:38.944591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:38.945337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:38.947862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:38.982564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 903/1802] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:38 2022] POST /identity/v3/users => generated 285 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:38.988651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:38.992118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:38.993365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9d673-20f8-4a21-a2a1-0b642fd2f369 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.000714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.001641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.001933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.002257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] PATH_INFO: `/v3/users/72bec2c71fed483496067dc790ac4740` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.003887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72bec2c71fed483496067dc790ac4740)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.005921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.008841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32283d-ed2a-4341-a307-c9053d377cd9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.019568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.020821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.021326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.022130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.023242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.025957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.045979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 904/1803] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:39.047817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.063010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056862c3-781e-4ff9-8c64-6eaa44badf30 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.076541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.077566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.080543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.081242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] PATH_INFO: `/v3/projects/5c79f56d80944b5ea1e39a743874d366/users/9c8ad710ba1e43ecba93f4ac1b57139f/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.103097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72bec2c71fed483496067dc790ac4740'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.107260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c79f56d80944b5ea1e39a743874d366, user_id=9c8ad710ba1e43ecba93f4ac1b57139f, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.110040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.126199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72bec2c71fed483496067dc790ac4740'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.127379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 905/1804] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:39 2022] PUT /identity/v3/projects/5c79f56d80944b5ea1e39a743874d366/users/9c8ad710ba1e43ecba93f4ac1b57139f/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:39.132455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.137514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72bec2c71fed483496067dc790ac4740'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.138349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72bec2c71fed483496067dc790ac4740'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.139655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72bec2c71fed483496067dc790ac4740'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.163655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:39.169327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 900/1805] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:38 2022] DELETE /identity/v3/users/72bec2c71fed483496067dc790ac4740 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:39.245056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875340cc-7f31-4dcd-835c-1f888a3b232f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.319127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc617c9-4fc3-41d1-a09f-4ab62937f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.320283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc617c9-4fc3-41d1-a09f-4ab62937f95b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.320830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc617c9-4fc3-41d1-a09f-4ab62937f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.321330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc617c9-4fc3-41d1-a09f-4ab62937f95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.323352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc617c9-4fc3-41d1-a09f-4ab62937f95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.326886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc617c9-4fc3-41d1-a09f-4ab62937f95b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.345231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 906/1806] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:39.347474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.348928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.364274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fac76-9f43-4b45-b8bb-e6ba73b67338 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.369864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.370793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.371109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.371363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] PATH_INFO: `/v3/projects/5c79f56d80944b5ea1e39a743874d366/users/9c8ad710ba1e43ecba93f4ac1b57139f/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.373525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c79f56d80944b5ea1e39a743874d366, user_id=9c8ad710ba1e43ecba93f4ac1b57139f, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.376589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.378967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:39.395565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 901/1807] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:39 2022] PUT /identity/v3/projects/5c79f56d80944b5ea1e39a743874d366/users/9c8ad710ba1e43ecba93f4ac1b57139f/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:39.413234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.413737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.414361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.444482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] MFA Rules not processed for user `9c8ad710ba1e43ecba93f4ac1b57139f`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:39.445775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.523627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 907/1808] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:39.534854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.537734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.538140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.539905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.627130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:39.642070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296910a-1589-4f2e-9438-280d7341069d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.647783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.648885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.649255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.649535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] PATH_INFO: `/v3/projects/38e234b58f424f55b0bc69bf985bd475` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.651609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e234b58f424f55b0bc69bf985bd475)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.651853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df13d99-6ad5-424c-8654-8a6058c05bff None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.653760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.663511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4b3c0-8c58-45b4-b791-2abae904c575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.665479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4b3c0-8c58-45b4-b791-2abae904c575 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.666602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4b3c0-8c58-45b4-b791-2abae904c575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.667138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4b3c0-8c58-45b4-b791-2abae904c575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.686290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4b3c0-8c58-45b4-b791-2abae904c575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:39.688983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4b3c0-8c58-45b4-b791-2abae904c575 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:39.714225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 908/1809] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:39.727392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e234b58f424f55b0bc69bf985bd475', 'request_id': 'req-7a7df627-680c-4aac-9236-29c762dff560'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.735233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e234b58f424f55b0bc69bf985bd475', 'request_id': 'req-7a7df627-680c-4aac-9236-29c762dff560'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.746747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e234b58f424f55b0bc69bf985bd475', 'request_id': 'req-7a7df627-680c-4aac-9236-29c762dff560'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.770202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] The token cache is being invalidate because project 38e234b58f424f55b0bc69bf985bd475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:39.770729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.771869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:39.774769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 902/1810] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:39 2022] DELETE /identity/v3/projects/38e234b58f424f55b0bc69bf985bd475 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:39.782510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.783046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.783518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.822111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:39.822111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.829961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.832242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.956824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7df627-680c-4aac-9236-29c762dff560 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.957709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 909/1811] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:39.962001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:39.963867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.964136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.965845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.965845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.967264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.970692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:39.980457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f0232-90f0-4024-b84f-073825fd2356 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:39.986457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:39.987298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:39.987686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:39.988146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:39.990472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:40.051848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:40.095669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.098489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01198a-1fb8-47e3-ba07-464cdba9ff61 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.099338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.103074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.110713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 903/1812] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:40.111873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.112169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.112485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.117862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 910/1813] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:40.141238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:40.143230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:40.162379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 904/1814] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:40.168987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:40.193437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf9e3c-bbb9-4afa-b260-f17f7b32096d service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:40.202967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:40.212919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.212919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.212919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.212919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:40.212919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.212919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.261001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 911/1815] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:01:40.269602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:40.286720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a06c9c-aa06-441d-be13-864c801afa98 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:40.309223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:40.310680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.311876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.312263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.314055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.321412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.379025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 905/1816] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:40 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:40.385974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:40.401506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddacde1-3d43-4165-8974-00fbc0db41a9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:40.505792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:40.506485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.506762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.507035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.508300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.510571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.528275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:40.536413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 912/1817] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:40 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:40.541725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:40.553925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655d00-63aa-4ee4-a109-d5fe4e34a667 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:40.561653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:40.562652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.563045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.563390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.564507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.566423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.588440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 906/1818] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:40.591289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:40.610867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d4f2-2de9-4a91-a74b-36666afd91f2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:40.619120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916634a-1263-4aed-9eab-1305c4b65d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:40.620304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916634a-1263-4aed-9eab-1305c4b65d78 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.620862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916634a-1263-4aed-9eab-1305c4b65d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.621374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916634a-1263-4aed-9eab-1305c4b65d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.622845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916634a-1263-4aed-9eab-1305c4b65d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.626101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916634a-1263-4aed-9eab-1305c4b65d78 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.642016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 913/1819] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:40.648550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:40.667547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5643f9-d44b-435a-bfe7-3b454b1da0c4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:40.675647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:40.677239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.677856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.678429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] PATH_INFO: `/v3/projects/2442a0daee7d42e9adf18441b18d2a51/users/ffff7c8613e54cefaec7f8683d8c53d8/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.706381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2442a0daee7d42e9adf18441b18d2a51, user_id=ffff7c8613e54cefaec7f8683d8c53d8, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:40.712024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:40.737686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 907/1820] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:40 2022] PUT /identity/v3/projects/2442a0daee7d42e9adf18441b18d2a51/users/ffff7c8613e54cefaec7f8683d8c53d8/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:40.750348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:40.751268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:40.752001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:40.787956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] MFA Rules not processed for user `ffff7c8613e54cefaec7f8683d8c53d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:40.789242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:40.892206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 914/1821] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:40.909152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:41.082736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d876ea9-ccb9-446a-9141-aaa107a264bf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:41.089544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:41.090259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.091378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.091378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.091727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:41.093661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:41.139669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:41.147725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 908/1822] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:40 2022] POST /identity/v3/projects => generated 381 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:41.157459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d1ddf-42ab-4878-bde6-dacb9649d748 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:41.245633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:41.246792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.247296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.247821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.249513 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:41.252960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:41.295917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 915/1823] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:41 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:41.296353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:41.311376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fd3ab-11d9-4263-a7ed-99149d4b2648 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:41.316630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:41.317324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.317608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.317889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.318636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:41.320259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:41.336423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 909/1824] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:01:41.342949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:41.362824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f6b2-ab7a-4bfc-a8c9-8df78bf93d4f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:41.371169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:41.372668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.373213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.373578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] PATH_INFO: `/v3/projects/90411d027ff04cfcace48be8f0d6b58e/users/80963056bb444b29bc21c84fd6b1117c/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.404661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90411d027ff04cfcace48be8f0d6b58e, user_id=80963056bb444b29bc21c84fd6b1117c, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:41.412571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:41.448597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 916/1825] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:41 2022] PUT /identity/v3/projects/90411d027ff04cfcace48be8f0d6b58e/users/80963056bb444b29bc21c84fd6b1117c/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:41.456792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:41.502782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5312222-5dee-4385-a564-c395b7d6d1b1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:41.680543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:41.709946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f1cc-369c-487a-9269-cb1f22a2dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:41.714380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f1cc-369c-487a-9269-cb1f22a2dd76 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.714916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f1cc-369c-487a-9269-cb1f22a2dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.715625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f1cc-369c-487a-9269-cb1f22a2dd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.716584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f1cc-369c-487a-9269-cb1f22a2dd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:41.720580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f1cc-369c-487a-9269-cb1f22a2dd76 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:41.757931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 910/1826] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:41.759362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:41.801602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942142d4-5909-4f29-a5ed-d0b82828a649 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:41.821488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:41.823226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.823640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.824082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] PATH_INFO: `/v3/projects/90411d027ff04cfcace48be8f0d6b58e/users/80963056bb444b29bc21c84fd6b1117c/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.834159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=90411d027ff04cfcace48be8f0d6b58e, user_id=80963056bb444b29bc21c84fd6b1117c, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:41.841755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:41.888645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 917/1827] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:41 2022] PUT /identity/v3/projects/90411d027ff04cfcace48be8f0d6b58e/users/80963056bb444b29bc21c84fd6b1117c/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:41.888846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:41.888846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:41.889089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:41.933121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] MFA Rules not processed for user `80963056bb444b29bc21c84fd6b1117c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:41.934432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:42.062901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 911/1828] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:42.071892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:42.139221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:42.211894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d599e-81af-4fd9-ae5a-c6149ea9b0dc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:42.221271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486eb1f-7359-4c36-b801-431d4eea47e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:42.222415 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486eb1f-7359-4c36-b801-431d4eea47e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:42.222876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486eb1f-7359-4c36-b801-431d4eea47e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:42.223328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486eb1f-7359-4c36-b801-431d4eea47e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:42.249835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486eb1f-7359-4c36-b801-431d4eea47e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:42.252087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486eb1f-7359-4c36-b801-431d4eea47e2 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:42.270613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 918/1829] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:42.781029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:42.921147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8d7a9-39cf-4711-90f4-e954bc363965 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:42.930234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:42.931778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:42.932257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:42.932764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] PATH_INFO: `/v3/users/ffff7c8613e54cefaec7f8683d8c53d8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:42.937370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffff7c8613e54cefaec7f8683d8c53d8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:42.943969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:43.047624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffff7c8613e54cefaec7f8683d8c53d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.062465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffff7c8613e54cefaec7f8683d8c53d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.071556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffff7c8613e54cefaec7f8683d8c53d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.072730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffff7c8613e54cefaec7f8683d8c53d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.073407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffff7c8613e54cefaec7f8683d8c53d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.075054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:43.104607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 912/1830] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:42 2022] DELETE /identity/v3/users/ffff7c8613e54cefaec7f8683d8c53d8 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:43.107578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:43.108421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:43.108421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:43.138907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:43.141273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:43.222116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 919/1831] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:43.232769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:43.234232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:43.313616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28944d4c-9ace-4c9a-ab44-96b948df61ec admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:43.319906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:43.321087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:43.321677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:43.322786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:43.334282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:43.335750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:43.347120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 913/1832] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:43.669160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:43.670910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:43.697216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c262-a3b8-4b29-a9ad-7f0f5eaf1c5a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:43.703344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:43.704471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:43.704830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:43.705223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] PATH_INFO: `/v3/projects/2442a0daee7d42e9adf18441b18d2a51` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:43.707064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2442a0daee7d42e9adf18441b18d2a51)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:43.710255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:43.769746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2442a0daee7d42e9adf18441b18d2a51', 'request_id': 'req-e69bda2d-b04d-473c-ad22-b20c3910439d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.777669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2442a0daee7d42e9adf18441b18d2a51', 'request_id': 'req-e69bda2d-b04d-473c-ad22-b20c3910439d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.786306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2442a0daee7d42e9adf18441b18d2a51', 'request_id': 'req-e69bda2d-b04d-473c-ad22-b20c3910439d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.799142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] The token cache is being invalidate because project 2442a0daee7d42e9adf18441b18d2a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:43.799579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.800369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:43.802208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 920/1833] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:43 2022] DELETE /identity/v3/projects/2442a0daee7d42e9adf18441b18d2a51 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:43.807097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:43.826093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b797a-e432-48a0-8606-25c4c5596770 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:43.920530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:43.922648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:43.923460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:43.924175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] PATH_INFO: `/v3/users/80963056bb444b29bc21c84fd6b1117c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:43.926940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80963056bb444b29bc21c84fd6b1117c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:43.932453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:44.027519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80963056bb444b29bc21c84fd6b1117c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:44.050837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80963056bb444b29bc21c84fd6b1117c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:44.061477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80963056bb444b29bc21c84fd6b1117c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:44.062363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80963056bb444b29bc21c84fd6b1117c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:44.063739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9277ed4-8622-4792-b2ae-341f517faad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80963056bb444b29bc21c84fd6b1117c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:44.081889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 914/1834] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:43 2022] DELETE /identity/v3/users/80963056bb444b29bc21c84fd6b1117c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:44.862472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:44.864428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:44.888528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bda2d-b04d-473c-ad22-b20c3910439d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:44.972326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:44.995661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:44.998025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:44.998570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:44.999077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] PATH_INFO: `/v3/projects/90411d027ff04cfcace48be8f0d6b58e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.001471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90411d027ff04cfcace48be8f0d6b58e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.005498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.087272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90411d027ff04cfcace48be8f0d6b58e', 'request_id': 'req-5a9ce5b2-488e-4def-b2d0-f197fea5213b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:45.122119 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90411d027ff04cfcace48be8f0d6b58e', 'request_id': 'req-5a9ce5b2-488e-4def-b2d0-f197fea5213b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:45.155214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90411d027ff04cfcace48be8f0d6b58e', 'request_id': 'req-5a9ce5b2-488e-4def-b2d0-f197fea5213b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:45.193102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] The token cache is being invalidate because project 90411d027ff04cfcace48be8f0d6b58e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:45.193615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:45.194784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5a9ce5b2-488e-4def-b2d0-f197fea5213b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:45.198831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 921/1835] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:44 2022] DELETE /identity/v3/projects/90411d027ff04cfcace48be8f0d6b58e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:45.204563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.205788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.206082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.228603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:45.229981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:45.241908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:45.319759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 915/1836] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:45.323005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.323005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.323273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.345858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:45.347147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:45.356742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 922/1837] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:45.361298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:45.373589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d2095-1e38-4406-b6ed-505d82d2456b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:45.379369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:45.380174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.380527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.380802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.381582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:45.382235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.384135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.402422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 916/1838] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:01:45.408359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:45.427922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edbac8-9b03-46fe-822b-0aee54deb5ed None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:45.436702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:45.438114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.438810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.439478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.441238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.445040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.499851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 923/1839] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:45 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:45.501140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:45.514835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18402b1-1ada-4008-ab54-d716661d4c2e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:45.631983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:45.633979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.635566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.636301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.638732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.645054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.692853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 917/1840] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:45 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:45.694336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:45.717897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a8651-ce6a-4833-a51e-20ccb140cfc7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:45.733110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:45.734609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.735086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.735561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.737035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.740263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.762803 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 924/1841] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:45.768036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:45.787928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f446b-6f4b-4bec-8af2-9ecc754c78df admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:45.796866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806710b6-7faf-4e26-a336-ff357af662a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:45.797751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806710b6-7faf-4e26-a336-ff357af662a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.801005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806710b6-7faf-4e26-a336-ff357af662a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.801590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806710b6-7faf-4e26-a336-ff357af662a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.803143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806710b6-7faf-4e26-a336-ff357af662a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.806510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806710b6-7faf-4e26-a336-ff357af662a3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.836726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 918/1842] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:45.839250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:45.856465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06b2b6-7086-41cc-956d-c069c0b3f200 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:45.864793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:45.866336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.866940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.867447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] PATH_INFO: `/v3/projects/682679f43f134426ab64b48f1d5bed12/users/21280df256be4d748610b05c6caef85a/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:45.898144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=682679f43f134426ab64b48f1d5bed12, user_id=21280df256be4d748610b05c6caef85a, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:45.902229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:45.919270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 925/1843] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:45 2022] PUT /identity/v3/projects/682679f43f134426ab64b48f1d5bed12/users/21280df256be4d748610b05c6caef85a/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:45.939564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:45.940737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:45.941174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.002456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] MFA Rules not processed for user `21280df256be4d748610b05c6caef85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:46.004942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:46.102489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 919/1844] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:46.108498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.206828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1fb25-bf4e-42c9-8b65-cc6d7f43e6c7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.216115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.217262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.217736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.218176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.219497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:46.223464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:46.268011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 926/1845] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:46 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:46.273229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.286405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136f20-6b2a-440a-ba64-4779647df805 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.291681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:46.370612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.371829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.372474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.372993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.374356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:46.378323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:46.410760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 920/1846] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:46 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:46.416327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.430512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9a852-8035-4eb7-8999-20b7c0556827 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.437367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.438135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.438468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.438752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.440656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:46.442504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:46.446050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:46.458961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 927/1847] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:46.470363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.490727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39b9fd-42e7-483f-8952-9f7c75a607b3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.500261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.501887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.502399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.502932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] PATH_INFO: `/v3/projects/da4a891053ae4648a9979c1f46bda75e/users/c406081c0737422d89105e3a008842ce/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.526083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4a891053ae4648a9979c1f46bda75e, user_id=c406081c0737422d89105e3a008842ce, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:46.529219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:46.550344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 921/1848] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:46 2022] PUT /identity/v3/projects/da4a891053ae4648a9979c1f46bda75e/users/c406081c0737422d89105e3a008842ce/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:46.557180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.569659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3b221-5bef-4b03-9374-d3d097bf017e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.635211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d834d72-4eb7-4c7b-9da0-7671c1d8ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.635910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d834d72-4eb7-4c7b-9da0-7671c1d8ff92 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.636195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d834d72-4eb7-4c7b-9da0-7671c1d8ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.636513 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d834d72-4eb7-4c7b-9da0-7671c1d8ff92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.637299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d834d72-4eb7-4c7b-9da0-7671c1d8ff92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:46.639070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d834d72-4eb7-4c7b-9da0-7671c1d8ff92 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:46.651073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 928/1849] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:01:46.657520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.668789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee996ff8-3be7-4f02-837f-c598f6991c74 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.674667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.675700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.675924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.676200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] PATH_INFO: `/v3/projects/da4a891053ae4648a9979c1f46bda75e/users/c406081c0737422d89105e3a008842ce/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.678456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4a891053ae4648a9979c1f46bda75e, user_id=c406081c0737422d89105e3a008842ce, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:46.681295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:46.694033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 922/1850] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:46 2022] PUT /identity/v3/projects/da4a891053ae4648a9979c1f46bda75e/users/c406081c0737422d89105e3a008842ce/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:46.705937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5189f8-1894-4d4c-91cf-b03e084035bd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.706224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5189f8-1894-4d4c-91cf-b03e084035bd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.706502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5189f8-1894-4d4c-91cf-b03e084035bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:46.738525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8c5189f8-1894-4d4c-91cf-b03e084035bd None None] MFA Rules not processed for user `c406081c0737422d89105e3a008842ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:46.739883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8c5189f8-1894-4d4c-91cf-b03e084035bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:46.875240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 929/1851] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:46.892712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:46.894773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:46.973977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbef491-a39e-4bcc-9229-06931b180169 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:46.982478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668980e3-c550-4008-84ba-dfc58c137384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:46.983773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668980e3-c550-4008-84ba-dfc58c137384 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:46.984356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668980e3-c550-4008-84ba-dfc58c137384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:46.985218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668980e3-c550-4008-84ba-dfc58c137384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:47.168921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668980e3-c550-4008-84ba-dfc58c137384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:47.171998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668980e3-c550-4008-84ba-dfc58c137384 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:47.202029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 923/1852] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:51.558800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dda0a7-2032-4a57-af81-f95192d528f3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:51.559352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dda0a7-2032-4a57-af81-f95192d528f3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:51.559721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dda0a7-2032-4a57-af81-f95192d528f3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:51.562423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 930/1853] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:01:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:01:51.574926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:51.575388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:51.575600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:51.627048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:51.630879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:51.669971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:51.895425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 924/1854] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:52.553174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dda0a7-2032-4a57-af81-f95192d528f3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:52.558355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0dda0a7-2032-4a57-af81-f95192d528f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:52.601963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dda0a7-2032-4a57-af81-f95192d528f3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:52.609514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993bf4-8a84-491d-923b-b9c73644b9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:52.610427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993bf4-8a84-491d-923b-b9c73644b9d6 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:52.610980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993bf4-8a84-491d-923b-b9c73644b9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:52.611532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993bf4-8a84-491d-923b-b9c73644b9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:52.624341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993bf4-8a84-491d-923b-b9c73644b9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:52.626184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993bf4-8a84-491d-923b-b9c73644b9d6 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:52.638921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 931/1855] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:54.626674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:54.627974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:54.763322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f17d6d-9780-4a2e-b4b0-71dd0209dc0b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:54.770308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:54.771374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:54.771784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:54.772153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] PATH_INFO: `/v3/users/f59649e585784a24bf7addb18e76a18c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:54.773978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59649e585784a24bf7addb18e76a18c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:54.777089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:54.852439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59649e585784a24bf7addb18e76a18c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:54.867670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59649e585784a24bf7addb18e76a18c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:54.878075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59649e585784a24bf7addb18e76a18c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:54.879341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59649e585784a24bf7addb18e76a18c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:54.880197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59649e585784a24bf7addb18e76a18c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:54.896500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 925/1856] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:54 2022] DELETE /identity/v3/users/f59649e585784a24bf7addb18e76a18c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:54.902401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:54.902978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:54.903188 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:54.923937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:54.925312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:54.997895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 932/1857] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:55.008461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:55.010713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:55.053034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:55.103380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550454a0-1532-43a4-8bca-64c09f6fd03d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:55.110263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:55.110943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:55.111235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:55.111539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:55.123973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:55.125600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:55.140023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 926/1858] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:01:55.551190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:55.552793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:55.561913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:55.566257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ccd2e-8eff-4b5d-ab5a-b6468b6bfafa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:55.573729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:55.574925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:55.575273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:55.575573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] PATH_INFO: `/v3/projects/a98b3700ace44b199bcf6bbdddb05cca` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:55.576917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98b3700ace44b199bcf6bbdddb05cca)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:55.579532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:55.654887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98b3700ace44b199bcf6bbdddb05cca', 'request_id': 'req-1928a315-88d4-4444-93ff-eb95e7fd7320'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.666907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98b3700ace44b199bcf6bbdddb05cca', 'request_id': 'req-1928a315-88d4-4444-93ff-eb95e7fd7320'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.684857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98b3700ace44b199bcf6bbdddb05cca', 'request_id': 'req-1928a315-88d4-4444-93ff-eb95e7fd7320'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.706925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] The token cache is being invalidate because project a98b3700ace44b199bcf6bbdddb05cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:55.707652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.708941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.713731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 933/1859] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:55 2022] DELETE /identity/v3/projects/a98b3700ace44b199bcf6bbdddb05cca => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:55.718263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:55.739256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30e086-62ef-4b10-9a3d-aff5b89bc44f service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:55.833159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:55.834490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:55.834869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:55.835334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] PATH_INFO: `/v3/users/2474f1efb2994a328d3b063de7daa41e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:55.836819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2474f1efb2994a328d3b063de7daa41e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:55.839640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:55.908721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2474f1efb2994a328d3b063de7daa41e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.934094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2474f1efb2994a328d3b063de7daa41e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.946142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2474f1efb2994a328d3b063de7daa41e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.947620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2474f1efb2994a328d3b063de7daa41e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.948493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2474f1efb2994a328d3b063de7daa41e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:55.974389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 927/1860] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:01:55 2022] DELETE /identity/v3/users/2474f1efb2994a328d3b063de7daa41e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:56.506347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:56.508663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:56.546695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a315-88d4-4444-93ff-eb95e7fd7320 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:56.713915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:56.718474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:56.777352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:56.782406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:56.802007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:56.802636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] PATH_INFO: `/v3/projects/1b332a1739824f08bdb8dbbe93b14cad` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:56.810769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b332a1739824f08bdb8dbbe93b14cad)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:56.820740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:56.923215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:56.950909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207a39c-f424-4976-ae62-1c0a43a42a16 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:56.969362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:56.972224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:56.974750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:56.976447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:56.988332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:57.085509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b332a1739824f08bdb8dbbe93b14cad', 'request_id': 'req-18cea408-76ca-45a3-8620-72358eb6ce73'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:57.140624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b332a1739824f08bdb8dbbe93b14cad', 'request_id': 'req-18cea408-76ca-45a3-8620-72358eb6ce73'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:57.194846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b332a1739824f08bdb8dbbe93b14cad', 'request_id': 'req-18cea408-76ca-45a3-8620-72358eb6ce73'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:57.265072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] The token cache is being invalidate because project 1b332a1739824f08bdb8dbbe93b14cad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:01:57.265576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:57.287142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18cea408-76ca-45a3-8620-72358eb6ce73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:01:57.290341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 934/1861] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:01:56 2022] DELETE /identity/v3/projects/1b332a1739824f08bdb8dbbe93b14cad => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:57.318821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:57.319328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:57.319833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:57.381563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:57.383032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:57.394942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:57.398852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59beed4-b43c-4188-9aa1-50a6f9c4e872 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:57.407874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:57.419434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 928/1862] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:01:57.541228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 935/1863] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:57.548315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:57.548855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:57.553135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:57.650861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:57.653098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:57.683565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:57.685364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 929/1864] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:01:57.696235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62ad08-c687-4572-829e-475f1c664a10 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:57.704975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:57.706536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:57.708294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:57.708916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:57.709919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:01:57.710503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:57.714175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:57.729296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 936/1865] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:01:57.734656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:57.783428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbd1a0-8151-4631-8761-c9780a44908a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:57.804855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:57.811557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:57.812259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:57.813307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:57.823484 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:57.827063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:57.875305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 930/1866] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:57 2022] POST /identity/v3/projects => generated 359 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:01:57.876529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:57.891521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6276f-269d-40aa-a3b5-d473fe023bea admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:57.990161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:57.990936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:57.991269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:57.991564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:57.992316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:57.994969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.034605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 937/1867] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:57 2022] POST /identity/v3/users => generated 280 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:58.035039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.048889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf406836-619b-45cb-994d-52e52b185637 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.061818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.063705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.065557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.066244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.068231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:58.073698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.098496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 931/1868] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:58.102933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.119272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a534dec-e041-4177-8cdc-8ec2cde1d4fc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.137463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ed436-27eb-4afd-ae4d-00158e7fee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.137618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed436-27eb-4afd-ae4d-00158e7fee43 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.138017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed436-27eb-4afd-ae4d-00158e7fee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.138402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed436-27eb-4afd-ae4d-00158e7fee43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.139339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed436-27eb-4afd-ae4d-00158e7fee43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:58.142220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed436-27eb-4afd-ae4d-00158e7fee43 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.168840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.189084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 938/1869] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:58.191788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc4fe-ebc0-4096-8835-71453d37c611 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.204961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.206930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.207275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.207561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] PATH_INFO: `/v3/projects/1efc768b36b4431a915063be54669644/users/dbe1d21f86414145b19ed7c287365ae1/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.248436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efc768b36b4431a915063be54669644, user_id=dbe1d21f86414145b19ed7c287365ae1, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:58.254533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.301386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 932/1870] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:58 2022] PUT /identity/v3/projects/1efc768b36b4431a915063be54669644/users/dbe1d21f86414145b19ed7c287365ae1/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:01:58.301802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.302309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.303008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.380010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] MFA Rules not processed for user `dbe1d21f86414145b19ed7c287365ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:58.381763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:58.531568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 939/1871] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:58 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:58.535883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.646558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc35ed8-0c5d-4cba-b431-11a6f18cf5ab admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.655769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.657238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.664517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.665966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.667009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:58.671749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.716485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 933/1872] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:01:58 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:58.720306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.739224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c972-a56a-4f5f-80af-749e74ea0651 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.757483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:58.835556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.836550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.837024 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.838031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.838983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:58.841225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.871897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 940/1873] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:01:58 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:01:58.875852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.890688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faa371-6487-48ed-94aa-f60b8d1d7ced admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.896632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.897756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.898424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.898975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:58.899894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:58.903230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:58.930499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 934/1874] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:58.935935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:58.961949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc569c3-12f9-4497-baf6-d1795ffd0e1a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:58.972655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:58.974743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:58.975458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:58.976139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] PATH_INFO: `/v3/projects/778965ae97fa45059787ae6016170a94/users/31567067db5e4d82a60a01cc5be5a1ea/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:59.043509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=778965ae97fa45059787ae6016170a94, user_id=31567067db5e4d82a60a01cc5be5a1ea, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:59.049791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:59.074693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 941/1875] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:58 2022] PUT /identity/v3/projects/778965ae97fa45059787ae6016170a94/users/31567067db5e4d82a60a01cc5be5a1ea/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:59.079065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:59.092569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b929ad-9d41-4552-a7f9-1af6ad703bbf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:59.097762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:01:59.220749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57fed3-94ac-4f27-9f0e-4d1696111a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:59.221746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fed3-94ac-4f27-9f0e-4d1696111a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:59.222159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fed3-94ac-4f27-9f0e-4d1696111a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:59.222670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fed3-94ac-4f27-9f0e-4d1696111a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:59.223591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57fed3-94ac-4f27-9f0e-4d1696111a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:59.225791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57fed3-94ac-4f27-9f0e-4d1696111a67 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:59.245262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 935/1876] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:01:59.269465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:59.284859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54749a1f-ffa4-4563-bdbd-3ca4a449cb3f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:59.305412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:59.307537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:59.308014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:59.308523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] PATH_INFO: `/v3/projects/778965ae97fa45059787ae6016170a94/users/31567067db5e4d82a60a01cc5be5a1ea/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:59.316581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778965ae97fa45059787ae6016170a94, user_id=31567067db5e4d82a60a01cc5be5a1ea, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:59.322236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:59.379075 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 942/1877] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:01:59 2022] PUT /identity/v3/projects/778965ae97fa45059787ae6016170a94/users/31567067db5e4d82a60a01cc5be5a1ea/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:01:59.382972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:59.383462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:59.383930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:59.469736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] MFA Rules not processed for user `31567067db5e4d82a60a01cc5be5a1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:01:59.472488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:59.695551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 936/1878] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:01:59.762864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:01:59.766066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:01:59.883601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3a7c5-cdfb-4d4c-bb2f-84a863bb2ac8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:01:59.889973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:01:59.891117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:01:59.891590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:01:59.892051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:01:59.908101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:01:59.910433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:01:59.925392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 943/1879] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:00.036808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:00.038585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:00.055178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec7bc0-f9d1-4dd8-b175-2dc21c9134d9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:00.060880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ab642-20f1-4e52-a6d7-6cc004b6a25e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:00.061785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ab642-20f1-4e52-a6d7-6cc004b6a25e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:00.062218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ab642-20f1-4e52-a6d7-6cc004b6a25e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:00.062639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ab642-20f1-4e52-a6d7-6cc004b6a25e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:00.167918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ab642-20f1-4e52-a6d7-6cc004b6a25e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:00.169799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ab642-20f1-4e52-a6d7-6cc004b6a25e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:00.183595 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 937/1880] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:00 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:04.866326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:04.978669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cabfe-8023-4b68-8f73-a48f737ecf32 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:04.987618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:04.988603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:04.989251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:04.989906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] PATH_INFO: `/v3/users/dbe1d21f86414145b19ed7c287365ae1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:04.991859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbe1d21f86414145b19ed7c287365ae1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:04.995640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:05.018263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:05.158637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbe1d21f86414145b19ed7c287365ae1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:05.203054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbe1d21f86414145b19ed7c287365ae1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:05.220361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbe1d21f86414145b19ed7c287365ae1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:05.257891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbe1d21f86414145b19ed7c287365ae1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:05.261108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbe1d21f86414145b19ed7c287365ae1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:05.270251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 944/1881] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:04 2022] DELETE /identity/v3/users/dbe1d21f86414145b19ed7c287365ae1 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:05.276561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:05.277160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:05.277160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:05.296304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:05.297579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:05.340061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:05.407755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 938/1882] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:05.414628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:05.416939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:05.560844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800e834-786f-4376-a55d-947de7135279 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:05.566046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99d4b7-498a-45f4-99c5-6d2708f75844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:05.566927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99d4b7-498a-45f4-99c5-6d2708f75844 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:05.567361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99d4b7-498a-45f4-99c5-6d2708f75844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:05.567736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99d4b7-498a-45f4-99c5-6d2708f75844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:05.593182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99d4b7-498a-45f4-99c5-6d2708f75844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:05.598571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99d4b7-498a-45f4-99c5-6d2708f75844 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:05.616021 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 945/1883] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:06.000636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:06.003102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:06.029353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f2ca6-0676-473a-b5b5-97fab40797d7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:06.038749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:06.041259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:06.041949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:06.042648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] PATH_INFO: `/v3/users/21280df256be4d748610b05c6caef85a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:06.045279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21280df256be4d748610b05c6caef85a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:06.049267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:06.186425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21280df256be4d748610b05c6caef85a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:06.238317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21280df256be4d748610b05c6caef85a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:06.270249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21280df256be4d748610b05c6caef85a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:06.273139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21280df256be4d748610b05c6caef85a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:06.277083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21280df256be4d748610b05c6caef85a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:06.313709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 939/1884] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:05 2022] DELETE /identity/v3/users/21280df256be4d748610b05c6caef85a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:06.318781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:06.319573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:06.320266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:06.371383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:06.373080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:06.494933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 946/1885] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:06.600959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:06.602620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:06.758064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fae5f1-3a23-4c55-9b4f-08867f490382 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:06.805236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:06.807617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:06.808437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:06.814304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:06.814846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:06.817693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:06.858591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e8ada-5728-48e1-8f2a-60e1d5e7783b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:06.872426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:06.872686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:06.874747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:06.875233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:06.875813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:06.876859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] PATH_INFO: `/v3/projects/1efc768b36b4431a915063be54669644` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:06.880017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1efc768b36b4431a915063be54669644)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:06.887887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:06.923757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 940/1886] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:06.994400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:07.043425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1efc768b36b4431a915063be54669644', 'request_id': 'req-73ab747e-1618-4e41-a841-31154f971a91'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.075070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1efc768b36b4431a915063be54669644', 'request_id': 'req-73ab747e-1618-4e41-a841-31154f971a91'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.109408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1efc768b36b4431a915063be54669644', 'request_id': 'req-73ab747e-1618-4e41-a841-31154f971a91'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.152128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] The token cache is being invalidate because project 1efc768b36b4431a915063be54669644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:07.152696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.153847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.163421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:07.164351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 947/1887] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:06 2022] DELETE /identity/v3/projects/1efc768b36b4431a915063be54669644 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:07.196500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ee47-926a-48d9-b66e-7b40eebc308a service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:07.308144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:07.351996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:07.354337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:07.355948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:07.356640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] PATH_INFO: `/v3/users/31567067db5e4d82a60a01cc5be5a1ea` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:07.358878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31567067db5e4d82a60a01cc5be5a1ea)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:07.362335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:07.419860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:07.422425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:07.441998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab747e-1618-4e41-a841-31154f971a91 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:07.448316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:07.449451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:07.449746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:07.450026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] PATH_INFO: `/v3/users/a0ddd975d874424cb50bdfcc13ac1289` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:07.451495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ddd975d874424cb50bdfcc13ac1289)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:07.454669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:07.480961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31567067db5e4d82a60a01cc5be5a1ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.508480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31567067db5e4d82a60a01cc5be5a1ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.521907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31567067db5e4d82a60a01cc5be5a1ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.522968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31567067db5e4d82a60a01cc5be5a1ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.524340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-04505fda-9d33-40ed-b2a6-88e7ad41dd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31567067db5e4d82a60a01cc5be5a1ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.543252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ddd975d874424cb50bdfcc13ac1289'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.547535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 941/1888] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:07 2022] DELETE /identity/v3/users/31567067db5e4d82a60a01cc5be5a1ea => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:07.567055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ddd975d874424cb50bdfcc13ac1289'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.577526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ddd975d874424cb50bdfcc13ac1289'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.597799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ddd975d874424cb50bdfcc13ac1289'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.598681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ddd975d874424cb50bdfcc13ac1289'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:07.609338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 948/1889] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:07 2022] DELETE /identity/v3/users/a0ddd975d874424cb50bdfcc13ac1289 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:07.616260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:07.621740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:07.621740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:07.657787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:07.660116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:07.777548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:07.779976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:07.866545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 942/1890] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:07.892732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:07.898390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:07.902578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de027b-4374-419c-8e76-dc8d68e62bc6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:07.914916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:07.917388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:07.917986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:07.918591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] PATH_INFO: `/v3/users/d7af34f911ed4eb0b7b7bd507b392ac8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:07.921844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7af34f911ed4eb0b7b7bd507b392ac8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:07.926195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:08.064895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=21,keystone:SELECT=77,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:08.086244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224047d3-ddb9-45a5-8075-3a6559ed7e45 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:08.107499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ad131-5c38-4f11-b11b-4ed7125fe267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:08.110272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ad131-5c38-4f11-b11b-4ed7125fe267 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:08.111110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ad131-5c38-4f11-b11b-4ed7125fe267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:08.112011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ad131-5c38-4f11-b11b-4ed7125fe267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:08.149022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ad131-5c38-4f11-b11b-4ed7125fe267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:08.152274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7af34f911ed4eb0b7b7bd507b392ac8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.154525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ad131-5c38-4f11-b11b-4ed7125fe267 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:08.176330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 943/1891] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:08.183449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7af34f911ed4eb0b7b7bd507b392ac8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.210420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7af34f911ed4eb0b7b7bd507b392ac8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.224669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7af34f911ed4eb0b7b7bd507b392ac8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.225573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7af34f911ed4eb0b7b7bd507b392ac8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.242052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 949/1892] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:07 2022] DELETE /identity/v3/users/d7af34f911ed4eb0b7b7bd507b392ac8 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:08.242335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:08.242335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:08.242619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:08.264056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:08.313177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:08.314757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:08.350666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:08.354580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:08.468538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd93a5-716a-41b0-b49f-659e33f576ef admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:08.478211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:08.478211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:08.478211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:08.478744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] PATH_INFO: `/v3/projects/778965ae97fa45059787ae6016170a94` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:08.482795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778965ae97fa45059787ae6016170a94)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:08.484350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:08.566367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 944/1893] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:08.568606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:08.572183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:08.595999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778965ae97fa45059787ae6016170a94', 'request_id': 'req-d678a02f-5b5d-4f01-bac9-239ad9294ac7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.600576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a4932-59d9-4f5a-a52c-6bef36bb0177 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:08.607741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778965ae97fa45059787ae6016170a94', 'request_id': 'req-d678a02f-5b5d-4f01-bac9-239ad9294ac7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.621546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778965ae97fa45059787ae6016170a94', 'request_id': 'req-d678a02f-5b5d-4f01-bac9-239ad9294ac7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.653365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] The token cache is being invalidate because project 778965ae97fa45059787ae6016170a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:08.655767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.657554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.662709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 950/1894] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:08 2022] DELETE /identity/v3/projects/778965ae97fa45059787ae6016170a94 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:08.669481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:08.673988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:08.737733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:08.738715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:08.739845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:08.740149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] PATH_INFO: `/v3/projects/682679f43f134426ab64b48f1d5bed12` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:08.741642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=682679f43f134426ab64b48f1d5bed12)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:08.744793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:08.800852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:08.828899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678a02f-5b5d-4f01-bac9-239ad9294ac7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:08.893770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '682679f43f134426ab64b48f1d5bed12', 'request_id': 'req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.906134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '682679f43f134426ab64b48f1d5bed12', 'request_id': 'req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.920784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '682679f43f134426ab64b48f1d5bed12', 'request_id': 'req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.947705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] The token cache is being invalidate because project 682679f43f134426ab64b48f1d5bed12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:08.948753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.949764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f9939fc-5ed5-47f2-96a2-a8a9811870b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:08.952293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 945/1895] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:08 2022] DELETE /identity/v3/projects/682679f43f134426ab64b48f1d5bed12 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:08.954135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:08.954655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:08.955145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:08.981061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:08.983896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.003564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:09.004879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:09.005496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:09.006067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:09.010897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.041463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:09.093466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 946/1896] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:09.094398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:09.097944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.115396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ab80c-026f-4e4a-852b-7d38d0824529 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:09.123988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:09.125659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:09.125918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:09.126384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] PATH_INFO: `/v3/projects/759f825197ce4ad29d4476f8cdcffe2d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:09.128108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=759f825197ce4ad29d4476f8cdcffe2d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:09.131238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:09.176569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:09.180003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:09.224464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 951/1897] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:09.226689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:09.237977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.241985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759f825197ce4ad29d4476f8cdcffe2d', 'request_id': 'req-81d7d35f-e249-4d81-91d7-e42e72427acc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.257520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759f825197ce4ad29d4476f8cdcffe2d', 'request_id': 'req-81d7d35f-e249-4d81-91d7-e42e72427acc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.273297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759f825197ce4ad29d4476f8cdcffe2d', 'request_id': 'req-81d7d35f-e249-4d81-91d7-e42e72427acc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.282479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517021af-3f3b-4aef-8440-5e236780ad9f service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:09.300723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] The token cache is being invalidate because project 759f825197ce4ad29d4476f8cdcffe2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:09.301250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.303034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81d7d35f-e249-4d81-91d7-e42e72427acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.309275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 947/1898] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:09 2022] DELETE /identity/v3/projects/759f825197ce4ad29d4476f8cdcffe2d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:09.314335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:09.314676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:09.314893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:09.357206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:09.358622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.482186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:09.487738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:09.488739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:09.489841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] PATH_INFO: `/v3/users/c406081c0737422d89105e3a008842ce` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:09.494571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c406081c0737422d89105e3a008842ce)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:09.502265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:09.518736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 948/1899] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:09.519253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:09.522648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.547300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02201-350c-4867-bea4-10da266f97bc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:09.564991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:09.565896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:09.566177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:09.566453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] PATH_INFO: `/v3/users/9c8ad710ba1e43ecba93f4ac1b57139f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:09.570153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8ad710ba1e43ecba93f4ac1b57139f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:09.572564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:09.675696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:09.689374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c406081c0737422d89105e3a008842ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.691141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8ad710ba1e43ecba93f4ac1b57139f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.704211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:09.728006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8ad710ba1e43ecba93f4ac1b57139f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.737708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c406081c0737422d89105e3a008842ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.750492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8ad710ba1e43ecba93f4ac1b57139f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.751385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8ad710ba1e43ecba93f4ac1b57139f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.752749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8ad710ba1e43ecba93f4ac1b57139f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.757136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c406081c0737422d89105e3a008842ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.781633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c406081c0737422d89105e3a008842ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.783383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c406081c0737422d89105e3a008842ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:09.807197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 949/1901] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:09 2022] DELETE /identity/v3/users/9c8ad710ba1e43ecba93f4ac1b57139f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:09.807197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:09.807996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 952/1901] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:09 2022] DELETE /identity/v3/users/c406081c0737422d89105e3a008842ce => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:09.810227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:09.988012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b57dd2-4c35-4d10-abd8-346653ec0623 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.071372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:10.072836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.073234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.073839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.077017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:10.077538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:10.081180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:10.100452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 950/1902] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:10.104696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:10.107050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:10.122728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749eeee7-cf9e-4b54-bb19-37ad7c03bafa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.131663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:10.132985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.133495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.134561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.136014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:10.140285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:10.203499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 953/1903] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:10 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:10.210448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:10.239415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73230685-600b-4a92-97bc-fb7df68b6fe7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.381157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:10.382380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.382869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.383404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.384762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:10.388763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:10.438109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 951/1904] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:10 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:10.442392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:10.463090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2149cc-29c0-412f-8539-a77730de7d46 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.482673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:10.483388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.483679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.483961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.484853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:10.495509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:10.524235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 954/1905] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:10.530252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:10.536824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:10.545565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475eb65-f0fa-4627-adfd-cc534caf9285 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.552566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a7620-cc33-4fff-81a8-edd98fa251a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:10.553483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a7620-cc33-4fff-81a8-edd98fa251a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.553790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a7620-cc33-4fff-81a8-edd98fa251a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.554595 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a7620-cc33-4fff-81a8-edd98fa251a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.555899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a7620-cc33-4fff-81a8-edd98fa251a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:10.558124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a7620-cc33-4fff-81a8-edd98fa251a3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:10.572476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 952/1906] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:10.576428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:10.589648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa63c-68c2-4651-91c3-4cca137643a1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.595347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:10.596624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.597025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.597408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] PATH_INFO: `/v3/projects/56911fce9f38438086d944d8a3c7ecf9/users/e5d2803ed85c4364a56617ffc9fd6b8c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.617331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=56911fce9f38438086d944d8a3c7ecf9, user_id=e5d2803ed85c4364a56617ffc9fd6b8c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:10.620868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:10.640598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 955/1907] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:10 2022] PUT /identity/v3/projects/56911fce9f38438086d944d8a3c7ecf9/users/e5d2803ed85c4364a56617ffc9fd6b8c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:10.651287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:10.651423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:10.651667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:10.686735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] MFA Rules not processed for user `e5d2803ed85c4364a56617ffc9fd6b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:10.688186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:10.833779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 953/1908] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:10.839073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:10.988744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd83b4-8a6c-4fd9-8a49-ae1d005c25ad admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:10.999328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.000966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.002325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.002986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.005804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.010824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.079824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:11.083109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.128002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 956/1909] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:10 2022] POST /identity/v3/projects => generated 369 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:11.130728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:11.137866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.144963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96741-aeb1-4dff-a094-d364a7b2e102 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:11.248103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.249109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.249866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.250159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] PATH_INFO: `/v3/projects/5c79f56d80944b5ea1e39a743874d366` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.251600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c79f56d80944b5ea1e39a743874d366)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.253690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.281160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:11.326122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce392e-f6c6-45af-9650-6912ac8868e1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:11.336037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.337501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.338861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.339154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] PATH_INFO: `/v3/projects/da4a891053ae4648a9979c1f46bda75e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.343325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da4a891053ae4648a9979c1f46bda75e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.345533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.381065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c79f56d80944b5ea1e39a743874d366', 'request_id': 'req-f688c390-b7b8-4e56-a785-60d4fab0b155'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.388152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c79f56d80944b5ea1e39a743874d366', 'request_id': 'req-f688c390-b7b8-4e56-a785-60d4fab0b155'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.402596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c79f56d80944b5ea1e39a743874d366', 'request_id': 'req-f688c390-b7b8-4e56-a785-60d4fab0b155'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.423338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] The token cache is being invalidate because project 5c79f56d80944b5ea1e39a743874d366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:11.423725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.424317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.430549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 954/1910] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:11 2022] DELETE /identity/v3/projects/5c79f56d80944b5ea1e39a743874d366 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:11.430549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:11.430549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.444285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:11.467235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da4a891053ae4648a9979c1f46bda75e', 'request_id': 'req-4f418f95-b849-430f-8053-ccff6f8b0b0e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.476420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da4a891053ae4648a9979c1f46bda75e', 'request_id': 'req-4f418f95-b849-430f-8053-ccff6f8b0b0e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.487710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da4a891053ae4648a9979c1f46bda75e', 'request_id': 'req-4f418f95-b849-430f-8053-ccff6f8b0b0e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.501253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] The token cache is being invalidate because project da4a891053ae4648a9979c1f46bda75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:11.501539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.502217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.504502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 957/1911] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:11 2022] DELETE /identity/v3/projects/da4a891053ae4648a9979c1f46bda75e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:11.505408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:11.506990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.547146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688c390-b7b8-4e56-a785-60d4fab0b155 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:11.585065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f418f95-b849-430f-8053-ccff6f8b0b0e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:11.590789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.591709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.591992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.592274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] PATH_INFO: `/v3/projects/356871afb8a742258c2128579e7416e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.593771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=356871afb8a742258c2128579e7416e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.595696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.661033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356871afb8a742258c2128579e7416e5', 'request_id': 'req-446ab66f-fa42-4314-bbb7-394e46564a4f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.662007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761918a-f208-4b70-93cd-508ff3336551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.663917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761918a-f208-4b70-93cd-508ff3336551 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.663917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761918a-f208-4b70-93cd-508ff3336551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.664556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761918a-f208-4b70-93cd-508ff3336551 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.666660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761918a-f208-4b70-93cd-508ff3336551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.669180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356871afb8a742258c2128579e7416e5', 'request_id': 'req-446ab66f-fa42-4314-bbb7-394e46564a4f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.670589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761918a-f208-4b70-93cd-508ff3336551 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.678494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356871afb8a742258c2128579e7416e5', 'request_id': 'req-446ab66f-fa42-4314-bbb7-394e46564a4f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.694111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] The token cache is being invalidate because project 356871afb8a742258c2128579e7416e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:11.694404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.695321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-446ab66f-fa42-4314-bbb7-394e46564a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:11.697774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 958/1912] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:11 2022] DELETE /identity/v3/projects/356871afb8a742258c2128579e7416e5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:11.702709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.702709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.702709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.704667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 955/1913] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:11 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:11.708932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.709775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.710862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.731577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:11.734070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.737187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:11.738650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.743938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:11.843181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 956/1914] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:11.844763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:11.854293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.863092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 959/1915] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:11.864589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:11.866694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.878240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bd47b-34c5-48ca-8cc7-a8ad6b086dc0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:11.888883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.890468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.890943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.891398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] PATH_INFO: `/v3/users/e3d527da08144ca3b11a1114df8b7c2b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.891601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c42bb8-5458-45fc-b878-486d98c928a7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:11.894051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d527da08144ca3b11a1114df8b7c2b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.897737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.899204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed35995-be39-4fe0-a413-b7ec7c29e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:11.900900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed35995-be39-4fe0-a413-b7ec7c29e302 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.901356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed35995-be39-4fe0-a413-b7ec7c29e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.901831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed35995-be39-4fe0-a413-b7ec7c29e302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.903137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed35995-be39-4fe0-a413-b7ec7c29e302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:11.906416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed35995-be39-4fe0-a413-b7ec7c29e302 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:11.928857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 960/1916] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:11.928857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546623ec-c434-41d5-b803-d5b281c12db7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.928857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546623ec-c434-41d5-b803-d5b281c12db7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.929224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546623ec-c434-41d5-b803-d5b281c12db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:11.963088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-546623ec-c434-41d5-b803-d5b281c12db7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:11.964936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-546623ec-c434-41d5-b803-d5b281c12db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:11.982908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 961/1917] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:11.985758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:11.986204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:11.986713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.005185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:12.008184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d527da08144ca3b11a1114df8b7c2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.022782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:12.025101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.033545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d527da08144ca3b11a1114df8b7c2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.041842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 962/1918] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:12.046498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.051795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d527da08144ca3b11a1114df8b7c2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.052810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d527da08144ca3b11a1114df8b7c2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.054023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d527da08144ca3b11a1114df8b7c2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.060440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3405f3-de15-48d6-b381-91c0b96f3f74 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.068965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.070765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.071008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 957/1919] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:11 2022] DELETE /identity/v3/users/e3d527da08144ca3b11a1114df8b7c2b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:12.071717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.072211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] PATH_INFO: `/v3/projects/002afc99b1a84c2386cf43f8561a800c/users/4da83d80be714c2d99b23b48f2d5d6a0/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.075016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.076433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.106492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002afc99b1a84c2386cf43f8561a800c, user_id=4da83d80be714c2d99b23b48f2d5d6a0, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.112435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.146190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 963/1920] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:12 2022] PUT /identity/v3/projects/002afc99b1a84c2386cf43f8561a800c/users/4da83d80be714c2d99b23b48f2d5d6a0/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:12.149362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.153279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.187518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b46ba7-3ac4-43a0-a573-de841e3d82c8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.271044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.272109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.272587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.273333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.274280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:12.275308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.277544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.298652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 958/1921] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:12.299885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.303130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.324528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd989af-fbcb-47f6-b50a-4341e453b9fc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.330279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.330996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.331572 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.332649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.333472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.336047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.348949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178e708-c5d5-4158-a7a5-5a2558bc0ea2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.351013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 959/1922] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:12.355652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.364260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.367027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.367821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.368924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2df7-2d1f-491d-98f0-cb1ababe6208 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.369129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.370645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:12.371479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.374844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.375162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.375464 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.375749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.375910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.376698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.378862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.416673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 960/1923] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:12 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:12.421564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.427602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 964/1924] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:12.429372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.469417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1c7b6-1281-4fb9-a9ab-fc5a03185810 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.473185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:12.525282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ace82f-2461-4dbf-8890-a719e06f9880 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.531527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.532588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.532975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.533260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] PATH_INFO: `/v3/projects/002afc99b1a84c2386cf43f8561a800c/users/4da83d80be714c2d99b23b48f2d5d6a0/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.539035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002afc99b1a84c2386cf43f8561a800c, user_id=4da83d80be714c2d99b23b48f2d5d6a0, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.543088 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.566968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:12.568565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 961/1925] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:12 2022] PUT /identity/v3/projects/002afc99b1a84c2386cf43f8561a800c/users/4da83d80be714c2d99b23b48f2d5d6a0/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:12.569440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.579487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.580195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.580804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.581129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.582047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.584452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.615759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 965/1926] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:12 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:12.616896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.620805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.635048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5182f65-cd14-4659-a621-ed27ab643286 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.710846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcb7f4-bb4f-4438-b879-a958b6444263 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:12.717457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e667f-baa2-4798-8109-c84950dcd64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.718345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e667f-baa2-4798-8109-c84950dcd64d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.718708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e667f-baa2-4798-8109-c84950dcd64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.719160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e667f-baa2-4798-8109-c84950dcd64d admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.719265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:12.720165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e667f-baa2-4798-8109-c84950dcd64d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.720755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.721304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.721705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] PATH_INFO: `/v3/projects/fabbfac1b93f45a48ca3f5c5ddc4a55a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.722379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e667f-baa2-4798-8109-c84950dcd64d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.724437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabbfac1b93f45a48ca3f5c5ddc4a55a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:12.728356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:12.757883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 962/1927] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:12 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:12.763384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:12.763754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:12.764267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:12.803377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] MFA Rules not processed for user `4da83d80be714c2d99b23b48f2d5d6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:12.805531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.853329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabbfac1b93f45a48ca3f5c5ddc4a55a', 'request_id': 'req-29574a2e-8526-4590-a11f-111a25e51f3f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.869858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabbfac1b93f45a48ca3f5c5ddc4a55a', 'request_id': 'req-29574a2e-8526-4590-a11f-111a25e51f3f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.884937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabbfac1b93f45a48ca3f5c5ddc4a55a', 'request_id': 'req-29574a2e-8526-4590-a11f-111a25e51f3f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.908829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 963/1928] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:12.910908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.913775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] The token cache is being invalidate because project fabbfac1b93f45a48ca3f5c5ddc4a55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:12.914228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.916007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:12.921322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 966/1929] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:12 2022] DELETE /identity/v3/projects/fabbfac1b93f45a48ca3f5c5ddc4a55a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:12.922823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:12.925386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:12.990820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:13.045627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e0d96-e7d4-4ad2-82b9-276d0fca057e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.051562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a7bd-5863-4a41-a0fb-320ad0a231cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.052276 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a7bd-5863-4a41-a0fb-320ad0a231cb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.052613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a7bd-5863-4a41-a0fb-320ad0a231cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.052900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a7bd-5863-4a41-a0fb-320ad0a231cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.053670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a7bd-5863-4a41-a0fb-320ad0a231cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.055365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a7bd-5863-4a41-a0fb-320ad0a231cb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.060074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574a2e-8526-4590-a11f-111a25e51f3f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.067930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 964/1930] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:13.069345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.069640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.069925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.076191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.078515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.079422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.080252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.082218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.087693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.092817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:13.094119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:13.104699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 965/1931] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:13.105311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.107586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:13.136674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 967/1932] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:12 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:13.137870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.139915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:13.150764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:13.156252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb3345-6630-48b3-ad2e-122ab6310d82 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.163780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3811bd-051a-4427-9e48-9bf30aedb3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.165471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3811bd-051a-4427-9e48-9bf30aedb3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.166639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3811bd-051a-4427-9e48-9bf30aedb3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.167901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3811bd-051a-4427-9e48-9bf30aedb3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.168845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3811bd-051a-4427-9e48-9bf30aedb3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.171967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3811bd-051a-4427-9e48-9bf30aedb3b2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.192813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 968/1933] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:13.194544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.194985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.195403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.229491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3709e-921b-4b04-bc77-c7b6d86b9074 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.235320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.236095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.236632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.237043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.239114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:13.239428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:13.241901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:13.263615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 969/1934] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:13.263615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.282767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2687e-c469-4998-8cff-aca2259d91a2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.299153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.299869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.300154 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.300502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.301270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.302958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.342721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 970/1935] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:13.344446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.350248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.351736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.365040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 966/1936] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:13.366091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.378926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f462d14-e7bc-4c46-8338-2b501e0a5586 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.386462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d697-9a6d-477e-bb85-f71b1c2a4d77 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.386736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.387660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.387968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.388425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.389568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:13.389950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.393446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.417271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.419442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.419950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.420691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] PATH_INFO: `/v3/projects/326c857a9d53412987dc9d4a3f68b05c/users/d6d2a12666a74bce8094a8d840c2eb6f/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.421834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 967/1937] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:13.422680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.439462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63987f-b30a-4441-9096-ce854ed3e56a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.447503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.449251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.449577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.450086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.451203 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.453581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.469949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 968/1938] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:13.469949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.505101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=326c857a9d53412987dc9d4a3f68b05c, user_id=d6d2a12666a74bce8094a8d840c2eb6f, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.507812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565509-dbea-4873-b81f-19979ebf46dd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.511210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.521795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767868ce-2c7b-462e-ad83-df82743749dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.523557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767868ce-2c7b-462e-ad83-df82743749dd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.524071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767868ce-2c7b-462e-ad83-df82743749dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.526459 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767868ce-2c7b-462e-ad83-df82743749dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.528095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767868ce-2c7b-462e-ad83-df82743749dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:13.532871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767868ce-2c7b-462e-ad83-df82743749dd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:13.548008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 971/1939] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:13 2022] PUT /identity/v3/projects/326c857a9d53412987dc9d4a3f68b05c/users/d6d2a12666a74bce8094a8d840c2eb6f/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:13.562080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.608779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 969/1940] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:13 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:13.613924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.614419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.614960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:13.695463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] MFA Rules not processed for user `d6d2a12666a74bce8094a8d840c2eb6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:13.698614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:13.734340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:13.756482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d87418-32c4-4ad9-ac22-c09cae7d7ecd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:13.907018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 970/1941] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:13.909181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:13.967608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:13.968600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:13.968893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:13.969171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] PATH_INFO: `/v3/projects/2401ff5e2d0743b6b7eb71b54d8f1fd2/users/8d8aa15809584af08a44d1600755041c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.019009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2401ff5e2d0743b6b7eb71b54d8f1fd2, user_id=8d8aa15809584af08a44d1600755041c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:14.035147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:14.098711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 972/1942] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:13 2022] PUT /identity/v3/projects/2401ff5e2d0743b6b7eb71b54d8f1fd2/users/8d8aa15809584af08a44d1600755041c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:14.100228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:14.102002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:14.209445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7eeb9-f3b2-4a0c-a944-ce0fa82aa971 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:14.254723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:14.285852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ddef2-d260-4183-89f9-05b7a25ae65a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:14.294933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d5cd5-b0bc-4ee1-8fdc-515ae34c12dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:14.296594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5cd5-b0bc-4ee1-8fdc-515ae34c12dd service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.297088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5cd5-b0bc-4ee1-8fdc-515ae34c12dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.297624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d5cd5-b0bc-4ee1-8fdc-515ae34c12dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.491927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d5cd5-b0bc-4ee1-8fdc-515ae34c12dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:14.497739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:14.498550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d5cd5-b0bc-4ee1-8fdc-515ae34c12dd service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:14.499618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.500210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.504794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.506246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:14.511322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:14.541707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 973/1943] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:14.544502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.548868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.548868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.585665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 971/1944] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:13 2022] POST /identity/v3/users => generated 296 bytes in 678 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:14.588228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:14.610907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add839ea-1f7b-4611-a9b9-82d5d672ee48 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:14.627802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:14.632319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.632319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.632319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.633158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:14.656449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:14.657422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] MFA Rules not processed for user `8d8aa15809584af08a44d1600755041c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:14.657422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:14.663341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 972/1945] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:14.663341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:14.679321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45267d0d-c8e1-499c-9bf3-125ba2605f49 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:14.685061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:14.685980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.686268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.686547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] PATH_INFO: `/v3/users/d6d2a12666a74bce8094a8d840c2eb6f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.688608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d2a12666a74bce8094a8d840c2eb6f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:14.690710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:14.765957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:14.797619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d2a12666a74bce8094a8d840c2eb6f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:14.824710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d2a12666a74bce8094a8d840c2eb6f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:14.837768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 974/1946] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:14.838871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:14.850411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d2a12666a74bce8094a8d840c2eb6f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:14.852062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d2a12666a74bce8094a8d840c2eb6f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:14.853289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d2a12666a74bce8094a8d840c2eb6f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:14.867993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a6577-75ba-4291-988b-c01ee4928d72 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:14.876847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3d868-4243-4496-b37a-3362ec422813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:14.878026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3d868-4243-4496-b37a-3362ec422813 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.878530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3d868-4243-4496-b37a-3362ec422813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.879084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3d868-4243-4496-b37a-3362ec422813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.880584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3d868-4243-4496-b37a-3362ec422813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:14.887382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 973/1947] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:14 2022] DELETE /identity/v3/users/d6d2a12666a74bce8094a8d840c2eb6f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:14.887636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df3d868-4243-4496-b37a-3362ec422813 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:14.887827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:14.887827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:14.922985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 975/1948] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:14.925712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:14.926308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:14.926872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:14.975413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:14.978406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:14.990742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e688149-5f8f-4f5a-9e96-f600d5830f51 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:15.000871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:15.002333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:15.003937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:15.004541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:15.012324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:15.113254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 976/1949] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:15.115413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:15.132876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:15.159523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f20e7-0a97-43f4-92db-e65e5d9a2b4a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:15.183979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:15.186197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:15.187492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:15.189474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:15.190175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:15.191064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] PATH_INFO: `/v3/projects/408c1af69a2d4e06a4dc9ab2e27c4e8d/users/a59a27424fe5446196236847799f61ea/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:15.203679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 974/1950] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:15.205372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:15.208057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:15.217947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:15.227220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=408c1af69a2d4e06a4dc9ab2e27c4e8d, user_id=a59a27424fe5446196236847799f61ea, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:15.233864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:15.276454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 977/1951] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:15 2022] PUT /identity/v3/projects/408c1af69a2d4e06a4dc9ab2e27c4e8d/users/a59a27424fe5446196236847799f61ea/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:15.276454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:15.276454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:15.354466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590f6b0-5911-42e7-907c-df58cc8c2083 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:15.401019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:15.428145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a1dba-dd89-4d8d-aed4-8f06f47525d5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:15.434805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:15.436667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:15.437020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:15.437383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] PATH_INFO: `/v3/users/8d8aa15809584af08a44d1600755041c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:15.439328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8aa15809584af08a44d1600755041c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:15.442948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:15.469201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e3e74-3a75-47fc-9ab0-9b16f1ab5cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:15.470011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3e74-3a75-47fc-9ab0-9b16f1ab5cea service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:15.470365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3e74-3a75-47fc-9ab0-9b16f1ab5cea service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:15.470808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3e74-3a75-47fc-9ab0-9b16f1ab5cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:15.492792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e3e74-3a75-47fc-9ab0-9b16f1ab5cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:15.503771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e3e74-3a75-47fc-9ab0-9b16f1ab5cea service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:15.517917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 975/1952] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:15.533728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:15.534232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:15.551411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:15.583496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8aa15809584af08a44d1600755041c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:15.626532 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8aa15809584af08a44d1600755041c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:15.638439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] MFA Rules not processed for user `a59a27424fe5446196236847799f61ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:15.640792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:15.643676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8aa15809584af08a44d1600755041c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:15.661810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8aa15809584af08a44d1600755041c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:15.665062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f5da6ff4-1c7e-42a0-8f1c-7bb231218798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8aa15809584af08a44d1600755041c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:15.691323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 978/1953] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:15 2022] DELETE /identity/v3/users/8d8aa15809584af08a44d1600755041c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:15.700426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:15.701178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:15.701846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:15.796846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:15.818949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:15.910426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 976/1954] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:15.918308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:15.922947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:16.001990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64ff47-b523-4170-9589-9feac15c1a7e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:16.111710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 979/1955] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:16.194712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:16.197645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:16.234725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:16.236132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.236606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.237044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.237993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:16.241791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:16.249359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:16.335939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 977/1956] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:15 2022] POST /identity/v3/projects => generated 391 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:16.339794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:16.412056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca12e09-efe0-41ec-903b-cfd99fa1184c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:16.459751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:16.519306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032cd10-fd71-4db6-a456-f674f527c644 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:16.551903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:16.552809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.553102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.553496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.554460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:16.555871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:16.556652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:16.557319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.557858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.558398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.588538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:16.594258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:16.605607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 978/1957] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:16 2022] POST /identity/v3/users => generated 302 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:16.606854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:16.608865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:16.623842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633af64b-9596-4519-8216-26e6e7794ebd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:16.630524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:16.631256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 980/1958] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:16.631705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.632002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.632288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] PATH_INFO: `/v3/projects/326c857a9d53412987dc9d4a3f68b05c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.634323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=326c857a9d53412987dc9d4a3f68b05c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:16.634682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:16.637046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:16.669037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3545-5216-4de2-80a6-2e330d4d4df6 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:16.719384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:16.720117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326c857a9d53412987dc9d4a3f68b05c', 'request_id': 'req-d9db97e1-705b-4c60-be8c-2af2446d1f82'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:16.720619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.721124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.721913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.724025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:16.726686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:16.732835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326c857a9d53412987dc9d4a3f68b05c', 'request_id': 'req-d9db97e1-705b-4c60-be8c-2af2446d1f82'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:16.748052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326c857a9d53412987dc9d4a3f68b05c', 'request_id': 'req-d9db97e1-705b-4c60-be8c-2af2446d1f82'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:16.762285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 981/1959] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:16.763734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:16.765447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] The token cache is being invalidate because project 326c857a9d53412987dc9d4a3f68b05c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:16.765747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:16.769248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d9db97e1-705b-4c60-be8c-2af2446d1f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:16.773182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 979/1960] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:16 2022] DELETE /identity/v3/projects/326c857a9d53412987dc9d4a3f68b05c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:16.780226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1273b-931d-47de-91c0-ea83934808ee None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.780870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1273b-931d-47de-91c0-ea83934808ee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.781281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1273b-931d-47de-91c0-ea83934808ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.790546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95445a2-1c01-44d6-a503-18a4a2f9b9f8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:16.822175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90f1273b-931d-47de-91c0-ea83934808ee None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:16.823941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90f1273b-931d-47de-91c0-ea83934808ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:16.915801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:16.917325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:16.918270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:16.920726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] PATH_INFO: `/v3/projects/0ba5106d772248e8a2a3e9c3c9cf3f86/users/0833e7d0f43c4c518d7b0fb6ac1956a1/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:16.958741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba5106d772248e8a2a3e9c3c9cf3f86, user_id=0833e7d0f43c4c518d7b0fb6ac1956a1, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:16.964238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:16.991360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 982/1961] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:16 2022] PUT /identity/v3/projects/0ba5106d772248e8a2a3e9c3c9cf3f86/users/0833e7d0f43c4c518d7b0fb6ac1956a1/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:16.999429 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.002643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.028598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 980/1962] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:17.036412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.037576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.040745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.068896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:17.070299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.112115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:17.158297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:17.159521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207aed72-1966-451d-aeb8-c09b7eb93124 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:17.170208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:17.171658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.172059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.172509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.173423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:17.175568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:17.188624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 981/1963] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:17.191661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.192664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 983/1964] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:17.194490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.197249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.219547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151b00c-6ef6-4799-95d5-63dbb10adda7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:17.220027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f4460-7198-456e-9374-ec043d2a36c3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:17.226477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:17.230005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:17.231986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.232531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.233054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] PATH_INFO: `/v3/users/e5d2803ed85c4364a56617ffc9fd6b8c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.233609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.234087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.234558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.235989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:17.236638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d2803ed85c4364a56617ffc9fd6b8c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:17.236837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:17.240495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:17.241619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:17.266015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 984/1965] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:17.267184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.282439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26077cff-71ee-432e-9d64-14f7a159c32f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:17.288297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:17.289276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.289568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.289855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] PATH_INFO: `/v3/projects/0ba5106d772248e8a2a3e9c3c9cf3f86/users/0833e7d0f43c4c518d7b0fb6ac1956a1/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.292999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba5106d772248e8a2a3e9c3c9cf3f86, user_id=0833e7d0f43c4c518d7b0fb6ac1956a1, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:17.295838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:17.314232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 985/1966] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:17 2022] PUT /identity/v3/projects/0ba5106d772248e8a2a3e9c3c9cf3f86/users/0833e7d0f43c4c518d7b0fb6ac1956a1/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:17.315873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.320589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.345925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d2803ed85c4364a56617ffc9fd6b8c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.365843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d2803ed85c4364a56617ffc9fd6b8c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.377563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d2803ed85c4364a56617ffc9fd6b8c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.378537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d2803ed85c4364a56617ffc9fd6b8c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.379365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d2803ed85c4364a56617ffc9fd6b8c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.402740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 982/1967] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:17 2022] DELETE /identity/v3/users/e5d2803ed85c4364a56617ffc9fd6b8c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:17.406811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.409933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.488706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d220f-8cb3-4ad7-8232-188832b1e00b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:17.567893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eaabb-f5ad-45f5-8ebf-2ea337d5d33a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:17.578678 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ea4f-c5dd-4ff8-bf4d-87e0c7083d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:17.579845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ea4f-c5dd-4ff8-bf4d-87e0c7083d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.580364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ea4f-c5dd-4ff8-bf4d-87e0c7083d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.580904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ea4f-c5dd-4ff8-bf4d-87e0c7083d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.582183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861ea4f-c5dd-4ff8-bf4d-87e0c7083d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:17.586578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861ea4f-c5dd-4ff8-bf4d-87e0c7083d36 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:17.589326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:17.590850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.591331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.591804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] PATH_INFO: `/v3/projects/2401ff5e2d0743b6b7eb71b54d8f1fd2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.594210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2401ff5e2d0743b6b7eb71b54d8f1fd2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:17.597676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:17.652497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 983/1968] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:17 2022] POST /identity/v3/projects => generated 391 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:17.652497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.652497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.652497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.706529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] MFA Rules not processed for user `0833e7d0f43c4c518d7b0fb6ac1956a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:17.709760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.751629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2401ff5e2d0743b6b7eb71b54d8f1fd2', 'request_id': 'req-4dbd4e84-6d32-4430-bc4b-a095a967b781'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.770399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2401ff5e2d0743b6b7eb71b54d8f1fd2', 'request_id': 'req-4dbd4e84-6d32-4430-bc4b-a095a967b781'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.783602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2401ff5e2d0743b6b7eb71b54d8f1fd2', 'request_id': 'req-4dbd4e84-6d32-4430-bc4b-a095a967b781'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.786349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:17.804959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:17.813152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] The token cache is being invalidate because project 2401ff5e2d0743b6b7eb71b54d8f1fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:17.813559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.814444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4dbd4e84-6d32-4430-bc4b-a095a967b781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:17.820701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 986/1969] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:17 2022] DELETE /identity/v3/projects/2401ff5e2d0743b6b7eb71b54d8f1fd2 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:17.824749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732a1e-9e58-4259-b093-6a72be1bf35b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.825681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732a1e-9e58-4259-b093-6a72be1bf35b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.826314 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e732a1e-9e58-4259-b093-6a72be1bf35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:17.865251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7e732a1e-9e58-4259-b093-6a72be1bf35b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:17.866711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7e732a1e-9e58-4259-b093-6a72be1bf35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.878044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 984/1970] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:17.879593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:17.881956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:17.987402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 987/1971] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:17.989921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:17.990417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:17.990907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.006533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618b14f-0c76-4921-8c25-53867a37c6e6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.013717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.014897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.015263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.015675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.016896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.019002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:18.019741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.020506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.037045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 988/1972] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:18.037684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.039351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.068917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 985/1973] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:17 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:18.069945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.075387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.176005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b2ff1-af0c-47da-8b0c-55ddfa51b9a8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.181290 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70079e-ebc9-4687-bc4e-b8a40dcac54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.181986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70079e-ebc9-4687-bc4e-b8a40dcac54a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.182274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70079e-ebc9-4687-bc4e-b8a40dcac54a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.182554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70079e-ebc9-4687-bc4e-b8a40dcac54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.190665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fa3e6-6383-43b9-8bb4-8e53bfcba45a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.194229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70079e-ebc9-4687-bc4e-b8a40dcac54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.195647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.195843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70079e-ebc9-4687-bc4e-b8a40dcac54a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.197156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.197544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.197913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.207859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 989/1974] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:18.209329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.209613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.210025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.211591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.214536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.230501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 986/1975] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:18.232854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:18.234255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.238872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.246009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 990/1976] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:18.250693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.261000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f456b6f-30f5-4c95-b921-c441b7b63f01 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.266234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.266999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f99838-7389-4f69-80ca-2a33ee30356c service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.267206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.267347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.267583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.268299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:18.268645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.270549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.277025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.278175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.278680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.279370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.280866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.286025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 991/1977] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:18.291734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.293860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.303200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e757f-3a57-415b-884e-35821b386fdd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.309192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.309956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.310239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.310545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.311457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.313675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.320534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 987/1978] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:18.322019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.325211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.350802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 992/1979] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:18 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:18.351755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.353136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0dc0-0bf2-48a1-9a4b-48aede96cc80 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.430025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:18.448583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.449749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.450241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.450747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.454310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.459697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cffd4-275b-4204-b498-33a34189a205 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.465087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.465863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.466161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.467133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.468038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.470025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.483045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 993/1980] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:18.483962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.495288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:18.497727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f86299-efbb-4ca5-8bc5-19705c82bd96 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.503876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.504614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.504904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.505177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.506042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.511947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.539692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 994/1981] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:18 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:18.544969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.558719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115a42-0be6-440a-9a3d-b78c4a6426db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.564437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.565386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.566613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.566613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] PATH_INFO: `/v3/projects/b4c59bd900864f59bdc3cd109fee1ae8/users/fd368a758a324546af3bc79b1909917a/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.584795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c59bd900864f59bdc3cd109fee1ae8, user_id=fd368a758a324546af3bc79b1909917a, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.588435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.605101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.608383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe767d10-ca03-42c8-a2df-3753f1402577 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.616020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 995/1982] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:18 2022] PUT /identity/v3/projects/b4c59bd900864f59bdc3cd109fee1ae8/users/fd368a758a324546af3bc79b1909917a/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:18.617422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.635750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 988/1983] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:18.639158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.639700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.640461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.724976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] MFA Rules not processed for user `fd368a758a324546af3bc79b1909917a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:18.726460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.755628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6d3b9-16c1-4655-825e-3e70d4582bd4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.764749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.765824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.766342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.766770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.767724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.772332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.795276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 996/1984] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:18.795276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.796110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:18.820929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d1d2-a8f2-43b4-ab8b-d9f1ae8d2293 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.830181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:18.831593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:18.832073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:18.832735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] PATH_INFO: `/v3/projects/56911fce9f38438086d944d8a3c7ecf9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:18.834888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56911fce9f38438086d944d8a3c7ecf9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:18.840454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:18.903262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 989/1985] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:18.904354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:18.917711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff5f70-1350-4c5d-9b75-ad4bcfcc26fd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:18.970349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56911fce9f38438086d944d8a3c7ecf9', 'request_id': 'req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:18.988684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56911fce9f38438086d944d8a3c7ecf9', 'request_id': 'req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.006152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56911fce9f38438086d944d8a3c7ecf9', 'request_id': 'req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.027116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:19.027485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] The token cache is being invalidate because project 56911fce9f38438086d944d8a3c7ecf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:19.027833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:19.027986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.028140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:19.028640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:19.029450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:19.031163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:19.037355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.044546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 997/1986] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:18 2022] DELETE /identity/v3/projects/56911fce9f38438086d944d8a3c7ecf9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:19.046785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 990/1987] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:19.046927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:19.051673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:19.054202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:19.057786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:19.086404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defd8f5-cacd-42e9-a233-90a711812ace admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:19.099990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:19.102227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:19.105064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:19.105646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:19.106114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] PATH_INFO: `/v3/users/4da83d80be714c2d99b23b48f2d5d6a0` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:19.111296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da83d80be714c2d99b23b48f2d5d6a0)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:19.114707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:19.186242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c43a0-3638-4aa9-95d6-9eb3fe5c1a33 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:19.190606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:19.190942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:19.191698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:19.192061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:19.192441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:19.194719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:19.221079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da83d80be714c2d99b23b48f2d5d6a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.272994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da83d80be714c2d99b23b48f2d5d6a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.295442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da83d80be714c2d99b23b48f2d5d6a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.296234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da83d80be714c2d99b23b48f2d5d6a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.297073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da83d80be714c2d99b23b48f2d5d6a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:19.324588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 991/1988] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:19 2022] DELETE /identity/v3/users/4da83d80be714c2d99b23b48f2d5d6a0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:19.325475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:19.334411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:19.421693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:19.425774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b668fc-5d72-4b68-ab72-2dd24d770fa4 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:19.447577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 998/1989] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:19.460927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6af72a-0031-4491-878b-8f0d44b346e1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:19.470760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:19.472348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:19.472840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:19.473300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] PATH_INFO: `/v3/projects/eb9b76b44b754456acb2c223a76d59f4/users/946310556d5f487798fcd82b5be8b3e9/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:19.508432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9b76b44b754456acb2c223a76d59f4, user_id=946310556d5f487798fcd82b5be8b3e9, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:19.515531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:19.543721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 992/1990] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:19 2022] PUT /identity/v3/projects/eb9b76b44b754456acb2c223a76d59f4/users/946310556d5f487798fcd82b5be8b3e9/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:19.563468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:19.563823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:19.564442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:19.612526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] MFA Rules not processed for user `946310556d5f487798fcd82b5be8b3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:19.614368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:19.741706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 999/1991] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:19.749665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:19.910983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd9b67-9f7f-4f03-b161-24495e9b0d65 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:19.920426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:19.921477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:19.921967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:19.922638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:19.923928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:19.928011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:19.992377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:19.994971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:20.025042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc09ac7-89f2-4344-84d5-1721567dcaf4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.025370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 993/1992] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:19 2022] POST /identity/v3/projects => generated 373 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:20.030306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.080760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:20.183390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.185316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.185813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.186309 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] PATH_INFO: `/v3/projects/002afc99b1a84c2386cf43f8561a800c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.189146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c819bd-e1ea-4c84-9fc8-b6799ca23eeb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.192815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=002afc99b1a84c2386cf43f8561a800c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.194948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.196051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.197045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.197609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.198428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.198868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.200671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.203436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:20.227513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 994/1993] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:20 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:20.233232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.251370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabd93f-2ed7-4188-9995-0610b06180e5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.260034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.261107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.261620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.262150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.263261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.266246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.285449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 995/1994] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:20.291061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.305395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002afc99b1a84c2386cf43f8561a800c', 'request_id': 'req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:20.314642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002afc99b1a84c2386cf43f8561a800c', 'request_id': 'req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:20.315075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a844c96-e4b4-4c16-9bda-ba2b31c8668c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.331477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002afc99b1a84c2386cf43f8561a800c', 'request_id': 'req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:20.348973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] The token cache is being invalidate because project 002afc99b1a84c2386cf43f8561a800c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:20.349446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:20.350205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dacf7ff3-b2d3-4aa9-a003-f6e4740dfa26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:20.352413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1000/1995] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:19 2022] DELETE /identity/v3/projects/002afc99b1a84c2386cf43f8561a800c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:20.358251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc76bec-ccba-43ca-a1dd-6674f30576fa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.358538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc76bec-ccba-43ca-a1dd-6674f30576fa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.358956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc76bec-ccba-43ca-a1dd-6674f30576fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.385344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4dc76bec-ccba-43ca-a1dd-6674f30576fa None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:20.386640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4dc76bec-ccba-43ca-a1dd-6674f30576fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:20.499007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.500842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.501253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.503150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] PATH_INFO: `/v3/projects/31d5bda55e6044ddab94ce68368b9200/users/c012e30895d34dc3b573ef7ab3c5b2e7/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.503381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1001/1996] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:20.512561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.514758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.515455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.534467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d5bda55e6044ddab94ce68368b9200, user_id=c012e30895d34dc3b573ef7ab3c5b2e7, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.538028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.559252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 996/1997] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:20 2022] PUT /identity/v3/projects/31d5bda55e6044ddab94ce68368b9200/users/c012e30895d34dc3b573ef7ab3c5b2e7/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:20.563815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:20.565101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:20.566486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.568627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:20.695022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1002/1998] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:20.695267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161cb45-7aff-4052-bb20-a39fcd5ce609 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.696746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.700905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.702005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.702469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.702946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.704269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.714273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.721422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b3668-9176-4254-924a-47c99597baa4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.726524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 997/1999] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:20.734544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.737369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.738488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.738959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.739440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.740888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:20.741437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.744511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.758330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99afee6-e42d-4e75-8bb0-7f2cde390c9a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.777617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1003/2000] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:20.778080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab5c4a-c747-4730-b841-be2534526b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.778080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab5c4a-c747-4730-b841-be2534526b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.778080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab5c4a-c747-4730-b841-be2534526b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.778080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab5c4a-c747-4730-b841-be2534526b17 admin admin] PATH_INFO: `/v3/projects/31d5bda55e6044ddab94ce68368b9200/users/c012e30895d34dc3b573ef7ab3c5b2e7/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.780095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.785874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab5c4a-c747-4730-b841-be2534526b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d5bda55e6044ddab94ce68368b9200, user_id=c012e30895d34dc3b573ef7ab3c5b2e7, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.789633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab5c4a-c747-4730-b841-be2534526b17 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.800544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ad70d-a615-498b-a3c5-b873eae4c319 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:20.820471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:20.821700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.822516 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.823018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.824555 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 998/2001] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:20 2022] PUT /identity/v3/projects/31d5bda55e6044ddab94ce68368b9200/users/c012e30895d34dc3b573ef7ab3c5b2e7/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:20.838569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:20.842726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:20.842931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:20.843263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:20.843709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:20.881333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:20.883039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:20.895989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1004/2002] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:20 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:20.903894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:20.958329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] MFA Rules not processed for user `c012e30895d34dc3b573ef7ab3c5b2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:20.961051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:21.100106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859552b-5855-448b-97e7-64bfbf384890 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:21.109330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:21.110496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.110981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.111464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.112955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:21.116774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:21.182819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 999/2003] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:21.190852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:21.194870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:21.195464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1005/2004] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:20 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:21.202160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:21.217560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf756c-539c-4760-849b-41ef5255da75 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:21.224571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:21.225441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.225733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.226010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.227975 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:21.230159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:21.252474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1006/2005] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:21.259871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:21.274240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67a9f7-86f4-4625-b3a7-d9401e8d34a3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:21.282500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:21.283573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.284474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.285015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.287385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:21.290266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:21.308700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1007/2006] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:21.315035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:21.332317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f64dc3-b88f-4204-b6b4-e7cdfdd23f17 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:21.349492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4b624-8ec4-49a2-9d3b-96a8b8d08705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:21.351123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4b624-8ec4-49a2-9d3b-96a8b8d08705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.351682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4b624-8ec4-49a2-9d3b-96a8b8d08705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.352182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4b624-8ec4-49a2-9d3b-96a8b8d08705 admin admin] PATH_INFO: `/v3/projects/820ab423187b45559bc764ea413e8da7/users/627a49b7340d4172927616427f50400f/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.367576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4fb9-3793-4fb6-b411-539ea49dd5e4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:21.375937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:21.377410 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.378058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.378659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.382536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:21.394406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4b624-8ec4-49a2-9d3b-96a8b8d08705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820ab423187b45559bc764ea413e8da7, user_id=627a49b7340d4172927616427f50400f, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:21.405791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4b624-8ec4-49a2-9d3b-96a8b8d08705 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:21.431715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1008/2007] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:21 2022] PUT /identity/v3/projects/820ab423187b45559bc764ea413e8da7/users/627a49b7340d4172927616427f50400f/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:21.445074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.446963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.446963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.512666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] MFA Rules not processed for user `627a49b7340d4172927616427f50400f`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:21.514170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:21.535812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:21.542241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:21.565817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1000/2008] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:21.646752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1009/2009] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:21.652482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:21.666428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a91e69-3777-4666-899d-cedc374e1262 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:21.775072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:21.849500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:21.851384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:21.851929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:21.854157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:21.854933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:21.863080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:21.946167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:21.967962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1001/2010] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:21 2022] POST /identity/v3/projects => generated 379 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:21.975840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dad17-7020-4107-98d7-8d1a51880c85 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:22.041522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:22.143079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:22.144911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.145712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.146586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.148432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:22.153926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:22.227262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1010/2011] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:21 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:22.230185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:22.278389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704bc69-4cbc-43f8-b8ef-a027ef5b7972 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:22.288588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:22.289561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.289852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.290260 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.291029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:22.292913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:22.313236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1002/2012] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:22.317650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:22.341313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486af46-a3f4-4052-90b3-28075bbbc987 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:22.350445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:22.351425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.351708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.351990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] PATH_INFO: `/v3/projects/1d7f21c3280b477bb7822485117d7253/users/fd3e2f37ec0a4cf99380e72ff236ac3c/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.379012 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7f21c3280b477bb7822485117d7253, user_id=fd3e2f37ec0a4cf99380e72ff236ac3c, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:22.385017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:22.408778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1011/2013] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:22 2022] PUT /identity/v3/projects/1d7f21c3280b477bb7822485117d7253/users/fd3e2f37ec0a4cf99380e72ff236ac3c/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:22.413682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:22.433429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c5745-2973-43ba-9987-1d8dd7d33446 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:22.529427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a6469-b360-406f-a796-5c372931479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:22.530498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6469-b360-406f-a796-5c372931479b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.530978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6469-b360-406f-a796-5c372931479b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.531376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a6469-b360-406f-a796-5c372931479b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.532692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a6469-b360-406f-a796-5c372931479b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:22.534439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a6469-b360-406f-a796-5c372931479b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:22.554184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1003/2014] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:22.555787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:22.570867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c62d1d-f5de-4b0c-a58d-d48443c16f81 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:22.578459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:22.579029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.579384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.579732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] PATH_INFO: `/v3/projects/1d7f21c3280b477bb7822485117d7253/users/fd3e2f37ec0a4cf99380e72ff236ac3c/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.582348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7f21c3280b477bb7822485117d7253, user_id=fd3e2f37ec0a4cf99380e72ff236ac3c, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:22.586007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:22.601181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1012/2015] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:22 2022] PUT /identity/v3/projects/1d7f21c3280b477bb7822485117d7253/users/fd3e2f37ec0a4cf99380e72ff236ac3c/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:22.617333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.617655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.617886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.650335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] MFA Rules not processed for user `fd3e2f37ec0a4cf99380e72ff236ac3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:22.651828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:22.733132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1004/2016] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:22.747317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:22.749458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:22.825184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d758c-9719-44bb-ab3d-c720bfb4656f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:22.829341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:22.830068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:22.830354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:22.830631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:22.842776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:22.844367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:22.857142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1013/2017] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:23.343781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:23.355472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c12d50-a248-44ab-b4a1-4a39781126e2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:23.363532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3e2f37ec0a4cf99380e72ff236ac3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7f21c3280b477bb7822485117d7253', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:23.364735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:23.365243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:23.365706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:23.366958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:23.367527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:23.371187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:23.392047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1005/2018] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:23.398193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:23.410178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58932836-fafe-4496-8e2d-9ffa2993b2dc service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:23.416972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3e2f37ec0a4cf99380e72ff236ac3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7f21c3280b477bb7822485117d7253', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:23.417896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:23.418362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:23.418827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:23.419671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:23.422001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:23.456127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1014/2019] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:23 2022] POST /identity/v3/projects => generated 353 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:23.463236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:23.481756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed8f2f-abba-4dc6-8e2e-bfecc68861b4 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:23.513714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:23.550991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3e2f37ec0a4cf99380e72ff236ac3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7f21c3280b477bb7822485117d7253', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:23.551672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:23.551960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:23.552279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:23.553067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:23.553387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:23.555120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:23.568303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1006/2020] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:23.630997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:23.643054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46502-5abe-41db-bb6b-a9f0e3f6346a tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:23.648958 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3e2f37ec0a4cf99380e72ff236ac3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7f21c3280b477bb7822485117d7253', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:23.650563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:23.651090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:23.651702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] PATH_INFO: `/v3/projects/31ccc4c62be44c9abae203268c2e7f00` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:23.653903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorizing `identity:delete_project(project_id=31ccc4c62be44c9abae203268c2e7f00)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:23.657839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:23.733414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ccc4c62be44c9abae203268c2e7f00', 'request_id': 'req-aff3dee7-88ce-4972-8b7d-d232dc703201'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:23.742479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ccc4c62be44c9abae203268c2e7f00', 'request_id': 'req-aff3dee7-88ce-4972-8b7d-d232dc703201'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:23.749108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:23.754585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ccc4c62be44c9abae203268c2e7f00', 'request_id': 'req-aff3dee7-88ce-4972-8b7d-d232dc703201'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:23.770779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] The token cache is being invalidate because project 31ccc4c62be44c9abae203268c2e7f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:23.771121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:23.771928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-aff3dee7-88ce-4972-8b7d-d232dc703201 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:23.773705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1015/2021] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:23 2022] DELETE /identity/v3/projects/31ccc4c62be44c9abae203268c2e7f00 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:24.028246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:24.030250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:24.113132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317769a6-4d0b-4649-bce9-b9d67f6684b9 tempest-VolumeQuotaClassesTest-749881721 tempest-VolumeQuotaClassesTest-749881721-project-admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:24.211606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:24.212854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:24.213243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:24.213703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] PATH_INFO: `/v3/users/627a49b7340d4172927616427f50400f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:24.215626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=627a49b7340d4172927616427f50400f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:24.218100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:24.302267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627a49b7340d4172927616427f50400f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:24.324341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627a49b7340d4172927616427f50400f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:24.329896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:24.338558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627a49b7340d4172927616427f50400f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:24.340476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627a49b7340d4172927616427f50400f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:24.343136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627a49b7340d4172927616427f50400f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:24.377962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1007/2022] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:24 2022] DELETE /identity/v3/users/627a49b7340d4172927616427f50400f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:24.384709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:24.385150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:24.385557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:24.409418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:24.411490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:24.490158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1016/2023] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:24.502540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:24.504112 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:24.588142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c4ad8-dee9-4779-8538-5e4a205d54e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:24.592225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:24.592964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:24.593249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:24.593525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:24.604673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:24.606181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:24.618661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1008/2024] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:25.018998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:25.020854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:25.041269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d2a19-7bb0-4445-b5e2-ecce1f30f6bc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:25.050533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:25.052137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:25.052668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:25.053175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] PATH_INFO: `/v3/projects/820ab423187b45559bc764ea413e8da7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:25.056041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] RBAC: Authorizing `identity:delete_project(project_id=820ab423187b45559bc764ea413e8da7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:25.057519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:25.129820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820ab423187b45559bc764ea413e8da7', 'request_id': 'req-27c88626-9309-4fa7-8b5f-62255c23b04b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.137536 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820ab423187b45559bc764ea413e8da7', 'request_id': 'req-27c88626-9309-4fa7-8b5f-62255c23b04b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.150497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820ab423187b45559bc764ea413e8da7', 'request_id': 'req-27c88626-9309-4fa7-8b5f-62255c23b04b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.172331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] The token cache is being invalidate because project 820ab423187b45559bc764ea413e8da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:25.172917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.173754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.179543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1017/2025] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:25 2022] DELETE /identity/v3/projects/820ab423187b45559bc764ea413e8da7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:25.182813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:25.200962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e360-78fd-4e9f-b927-9f44573f2130 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:25.339272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:25.342502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:25.342502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:25.342502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] PATH_INFO: `/v3/users/fd3e2f37ec0a4cf99380e72ff236ac3c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:25.344849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3e2f37ec0a4cf99380e72ff236ac3c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:25.349308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:25.467785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3e2f37ec0a4cf99380e72ff236ac3c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.521333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3e2f37ec0a4cf99380e72ff236ac3c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.542286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3e2f37ec0a4cf99380e72ff236ac3c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.546524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3e2f37ec0a4cf99380e72ff236ac3c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.550600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3e2f37ec0a4cf99380e72ff236ac3c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:25.597162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1009/2026] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:25 2022] DELETE /identity/v3/users/fd3e2f37ec0a4cf99380e72ff236ac3c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:26.068786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.071823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:26.097587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c88626-9309-4fa7-8b5f-62255c23b04b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.117207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.120953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:26.150768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:26.205211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.206471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.206889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.207260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] PATH_INFO: `/v3/projects/1d7f21c3280b477bb7822485117d7253` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.209230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7f21c3280b477bb7822485117d7253)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.211577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.226181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:26.231159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12525-f415-46fd-997f-b15da6c5a97a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.236967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.237879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.238168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.238452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] PATH_INFO: `/v3/users/a59a27424fe5446196236847799f61ea` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.239790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59a27424fe5446196236847799f61ea)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.241882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.272725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7f21c3280b477bb7822485117d7253', 'request_id': 'req-14da88c2-f92e-450d-8bac-5654eabbdc6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.283096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7f21c3280b477bb7822485117d7253', 'request_id': 'req-14da88c2-f92e-450d-8bac-5654eabbdc6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.295759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7f21c3280b477bb7822485117d7253', 'request_id': 'req-14da88c2-f92e-450d-8bac-5654eabbdc6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.307219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59a27424fe5446196236847799f61ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.309186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] The token cache is being invalidate because project 1d7f21c3280b477bb7822485117d7253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:26.309485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.310211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14da88c2-f92e-450d-8bac-5654eabbdc6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.311861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1018/2027] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:26 2022] DELETE /identity/v3/projects/1d7f21c3280b477bb7822485117d7253 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:26.319180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a92b76-788b-49d8-be5b-aaa4a54549fa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.319684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a92b76-788b-49d8-be5b-aaa4a54549fa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.320160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a92b76-788b-49d8-be5b-aaa4a54549fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.327920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59a27424fe5446196236847799f61ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.338041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59a27424fe5446196236847799f61ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.338824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59a27424fe5446196236847799f61ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.339552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec4c58e8-50c8-4317-9f07-08babdb78280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59a27424fe5446196236847799f61ea'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:26.355234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-18a92b76-788b-49d8-be5b-aaa4a54549fa None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:26.356957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18a92b76-788b-49d8-be5b-aaa4a54549fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:26.359803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1010/2028] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:26 2022] DELETE /identity/v3/users/a59a27424fe5446196236847799f61ea => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:26.363475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.364496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.364496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.389395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:26.390705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:26.436205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1019/2029] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:26.448633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.448907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.449244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.470545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:26.472166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:26.473526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1011/2030] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:26.481960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1020/2031] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:26.484637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.486665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:26.486983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.502567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8ec42-a42f-4814-a5a6-18a05f094ec5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.510349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.511339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.511801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.512222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.513411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:26.513918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.516855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.534990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1021/2032] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:26.536664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.540800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:26.548527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc625508-f851-4d55-8881-f226aaf0355a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.554004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.554904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.555188 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.555503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.556219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.558214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.579215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5196-0e8e-4013-a5af-df8bfe24538f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.584564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.585634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.586090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.586544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.596700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1022/2033] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:26 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:26.599884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.604787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.607523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.625710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1012/2034] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:26.693894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d167e-e2e8-4b0d-9990-a37299a20771 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.699721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.700493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.700796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.701068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.701792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.704223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.713211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:26.739868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1023/2035] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:26 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:26.741025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.754682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820db646-f85c-4efd-9be6-6e6be9410b22 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.764795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.765823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.766764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.767062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.769771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.771487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.793877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1013/2036] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:26.796240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.816605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e953ea-dc27-4b55-b433-0876e84d45d5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.822775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636d1c2-4ad3-42de-a3f4-c09cd0c43ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.824169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636d1c2-4ad3-42de-a3f4-c09cd0c43ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.824169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636d1c2-4ad3-42de-a3f4-c09cd0c43ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.824465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636d1c2-4ad3-42de-a3f4-c09cd0c43ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.825277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636d1c2-4ad3-42de-a3f4-c09cd0c43ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.827084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636d1c2-4ad3-42de-a3f4-c09cd0c43ad4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.843205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1024/2037] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:26.844849 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:26.879449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee0a21-f23b-42d0-ac3a-6f3a64e7916e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:26.885284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:26.886249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.886532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.886798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] PATH_INFO: `/v3/projects/894ea62d53ec4ecfade892d6014f7012/users/621d0d2532fd4e3c96012096f5a90f47/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:26.905832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=894ea62d53ec4ecfade892d6014f7012, user_id=621d0d2532fd4e3c96012096f5a90f47, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:26.909504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:26.932673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1014/2038] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:26 2022] PUT /identity/v3/projects/894ea62d53ec4ecfade892d6014f7012/users/621d0d2532fd4e3c96012096f5a90f47/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:26.959488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:26.959796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:26.960678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.008371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] MFA Rules not processed for user `621d0d2532fd4e3c96012096f5a90f47`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:27.010496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:27.077626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.080504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:27.132999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1025/2039] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:27.138102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.235399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56160a5-24bd-4297-9012-27a767f8af7b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.241353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.242473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.242807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.243200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] PATH_INFO: `/v3/projects/408c1af69a2d4e06a4dc9ab2e27c4e8d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.245154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=408c1af69a2d4e06a4dc9ab2e27c4e8d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.247401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.292969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5825661-8fa5-424b-91d5-0194e0bded81 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.298701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.299445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.299766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.300164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.300952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.302964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.323545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '408c1af69a2d4e06a4dc9ab2e27c4e8d', 'request_id': 'req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.333426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '408c1af69a2d4e06a4dc9ab2e27c4e8d', 'request_id': 'req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.333742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1026/2040] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:27 2022] POST /identity/v3/projects => generated 399 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:27.338387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.345316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '408c1af69a2d4e06a4dc9ab2e27c4e8d', 'request_id': 'req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.360255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] The token cache is being invalidate because project 408c1af69a2d4e06a4dc9ab2e27c4e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:27.360605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.361475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.363294 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1015/2041] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:27 2022] DELETE /identity/v3/projects/408c1af69a2d4e06a4dc9ab2e27c4e8d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:27.367702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.370141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:27.405398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:27.408761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:27.422228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4243c66-36c3-421f-86e7-7aaa5ab98283 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.428651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.429827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.430213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.431185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.432344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.435006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.454399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328d35b-8b00-4f7e-96b4-b3dd95e660a9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.459768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.460779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.461060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.461341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] PATH_INFO: `/v3/users/0833e7d0f43c4c518d7b0fb6ac1956a1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.462691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0833e7d0f43c4c518d7b0fb6ac1956a1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.466325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1027/2042] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:27 2022] POST /identity/v3/users => generated 306 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:27.466565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.467748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.470254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:27.487586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18928cc0-5396-4010-8095-75acbd856072 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.494897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.495605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.495907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.496450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.497211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.498921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.516356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1028/2043] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:27.521716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.528722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0833e7d0f43c4c518d7b0fb6ac1956a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.540928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618c82-53fe-4ed8-bb11-0678280f48ae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.548903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.550421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.550901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.551405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] PATH_INFO: `/v3/projects/4d64125bbffe4d619cb26897cfbec7c6/users/be764cc4456f40359fbd20183c2f1fc8/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.554325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0833e7d0f43c4c518d7b0fb6ac1956a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.567700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0833e7d0f43c4c518d7b0fb6ac1956a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.568582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0833e7d0f43c4c518d7b0fb6ac1956a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.569383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0833e7d0f43c4c518d7b0fb6ac1956a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:27.578121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d64125bbffe4d619cb26897cfbec7c6, user_id=be764cc4456f40359fbd20183c2f1fc8, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.581567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.585067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1016/2044] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:27 2022] DELETE /identity/v3/users/0833e7d0f43c4c518d7b0fb6ac1956a1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:27.602217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1029/2045] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:27 2022] PUT /identity/v3/projects/4d64125bbffe4d619cb26897cfbec7c6/users/be764cc4456f40359fbd20183c2f1fc8/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:27.608006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.609615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:27.701586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d01bb8-000b-4a94-a6eb-f28876481a11 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.761008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:27.782988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae12406-d706-45f8-8512-85011d30b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.784498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae12406-d706-45f8-8512-85011d30b10f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.785369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae12406-d706-45f8-8512-85011d30b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.786359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae12406-d706-45f8-8512-85011d30b10f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.787230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae12406-d706-45f8-8512-85011d30b10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.790003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae12406-d706-45f8-8512-85011d30b10f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.803397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1017/2046] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:27.808244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.820730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe716b-b915-46e5-8095-898701256e61 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:27.827515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:27.828956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.829511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.829980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] PATH_INFO: `/v3/projects/4d64125bbffe4d619cb26897cfbec7c6/users/be764cc4456f40359fbd20183c2f1fc8/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.833200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d64125bbffe4d619cb26897cfbec7c6, user_id=be764cc4456f40359fbd20183c2f1fc8, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:27.836621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:27.853874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1030/2047] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:27 2022] PUT /identity/v3/projects/4d64125bbffe4d619cb26897cfbec7c6/users/be764cc4456f40359fbd20183c2f1fc8/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:27.867065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:27.867402 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:27.867619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:27.909744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] MFA Rules not processed for user `be764cc4456f40359fbd20183c2f1fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:27.911192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:27.957954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:27.959918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.021382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1018/2048] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:28.033365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.035043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.047147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb596a-9b93-4baa-bdf4-410ef74201cd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.052521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.053413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.053723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.054025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] PATH_INFO: `/v3/projects/0ba5106d772248e8a2a3e9c3c9cf3f86` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.055344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba5106d772248e8a2a3e9c3c9cf3f86)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.057578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.106249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c97082-e4af-4e75-8e22-46be9986eb29 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.120500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba5106d772248e8a2a3e9c3c9cf3f86', 'request_id': 'req-771c6ac5-35c6-4083-881b-bc3f94144205'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:28.127638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba5106d772248e8a2a3e9c3c9cf3f86', 'request_id': 'req-771c6ac5-35c6-4083-881b-bc3f94144205'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:28.136802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba5106d772248e8a2a3e9c3c9cf3f86', 'request_id': 'req-771c6ac5-35c6-4083-881b-bc3f94144205'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:28.169242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] The token cache is being invalidate because project 0ba5106d772248e8a2a3e9c3c9cf3f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:28.169410 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:28.170505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-771c6ac5-35c6-4083-881b-bc3f94144205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:28.173151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1031/2049] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:27 2022] DELETE /identity/v3/projects/0ba5106d772248e8a2a3e9c3c9cf3f86 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:28.180267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3cdb-4ef7-4285-8cb5-b32fd65b6e25 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.180788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3cdb-4ef7-4285-8cb5-b32fd65b6e25 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.181590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3cdb-4ef7-4285-8cb5-b32fd65b6e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.189800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.190988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.192429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.192429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.195190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.210324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4a6f3cdb-4ef7-4285-8cb5-b32fd65b6e25 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:28.212635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4a6f3cdb-4ef7-4285-8cb5-b32fd65b6e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.244480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:28.305371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:28.334825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1032/2050] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:28.345372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.347168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.347812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.348313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.348812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.360279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1019/2051] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:28.375325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:28.377152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.393052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1033/2052] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:28.400006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.421326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a3cfd-fab4-47eb-b99e-c92444ee7fe9 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.436956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.438353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.438554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.438981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.440323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:28.440873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.445826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.481028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1020/2053] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:28.487463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.496432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.498668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.501526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60752781-0c42-4ab1-9dec-2b0701d5b6f9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.511471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.513633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.513633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.513949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.515529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.519427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.523859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc535836-5961-48a9-a796-66da38fec761 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.535069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.536029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.536029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.536300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.540692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.581322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1034/2054] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:28 2022] POST /identity/v3/projects => generated 389 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:28.586501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.672853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.676277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.695706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1021/2055] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:28.739675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b6ef-2141-486a-8ea8-35c21c1a7d9d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.751302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.752049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.752639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.752832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.754555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.757844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.787897 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1035/2056] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:28 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:28.800969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.832145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f10f0-c374-4db3-b950-b7f5a51c86d6 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.841600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.842991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.843577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.844141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.845611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.849067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.879080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1022/2057] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:28.886403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.901775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed7360-9865-458e-8315-b1f38673e94a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.908484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.909649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.910242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.910662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.911714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:28.913721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:28.940241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1036/2058] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:28.941959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.966829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:28.969928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:28.974114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1a5a-203b-4d7f-b680-3d1ebdeeb6d3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:28.984002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863c11-a376-47c5-93b6-8be8a81143de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:28.985747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863c11-a376-47c5-93b6-8be8a81143de admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:28.986537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863c11-a376-47c5-93b6-8be8a81143de admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:28.987177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863c11-a376-47c5-93b6-8be8a81143de admin admin] PATH_INFO: `/v3/projects/784bedc01b834c2980092e8d463c12e8/users/74b4d88cd8be4d0f816143cc6f8fce0a/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:28.995853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fdf20-cd9a-4ccf-9c6d-f4b8011e3546 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:29.012917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:29.014570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.015114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.015671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] PATH_INFO: `/v3/users/946310556d5f487798fcd82b5be8b3e9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.018467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=946310556d5f487798fcd82b5be8b3e9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.023505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.023735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a863c11-a376-47c5-93b6-8be8a81143de admin admin] RBAC: Authorizing `identity:create_grant(project_id=784bedc01b834c2980092e8d463c12e8, user_id=74b4d88cd8be4d0f816143cc6f8fce0a, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.032117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a863c11-a376-47c5-93b6-8be8a81143de admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.068977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1023/2059] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:28 2022] PUT /identity/v3/projects/784bedc01b834c2980092e8d463c12e8/users/74b4d88cd8be4d0f816143cc6f8fce0a/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:29.087529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.087997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.088478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.159652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946310556d5f487798fcd82b5be8b3e9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:29.187414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] MFA Rules not processed for user `74b4d88cd8be4d0f816143cc6f8fce0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:29.191407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:29.193202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946310556d5f487798fcd82b5be8b3e9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:29.209980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:29.219742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946310556d5f487798fcd82b5be8b3e9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:29.241979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946310556d5f487798fcd82b5be8b3e9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:29.245438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-15d328fe-880f-4da3-ad9d-649280ed8a68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946310556d5f487798fcd82b5be8b3e9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:29.280125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1037/2060] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:28 2022] DELETE /identity/v3/users/946310556d5f487798fcd82b5be8b3e9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:29.286759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.287248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.287757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.325204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:29.327471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:29.341820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1024/2061] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:29.349711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:29.353227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:29.411619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:29.444842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1038/2062] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:29.459665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:29.462621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:29.500220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba3fc-cad3-4d0c-86fb-54654f48e925 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:29.510703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:29.511578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.512060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.512675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.513703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.518523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.564712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1025/2063] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:29 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:29.571962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:29.575668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9bf32-1f4a-43e3-b62b-9371d119abbd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:29.661718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:29.662528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.663000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.663512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.725892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc107f0-868f-4aa5-b43d-39e4209d93df admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:29.754652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:29.755839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.756338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.756858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.758156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.762107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.807335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:29.815058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.818446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.820239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1026/2064] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:29 2022] POST /identity/v3/users => generated 300 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:29.826499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:29.846781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1039/2065] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:29.848833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6f62-f2a4-4623-913b-89ffb2b21c4b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:29.855949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:29.857110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.857508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.857715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.858615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.860375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.874240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1027/2066] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:29.878771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:29.891949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d808bb-ea36-4743-84a9-82213c2f09e3 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:29.898582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:29.900526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:29.901050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:29.901212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] PATH_INFO: `/v3/projects/bb6d08afb1a34e529808cbfd98260d39/users/22e3c936aae542d796147228db81b39f/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:29.941403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6d08afb1a34e529808cbfd98260d39, user_id=22e3c936aae542d796147228db81b39f, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:29.950583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:29.995807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:29.997027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1040/2067] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:29 2022] PUT /identity/v3/projects/bb6d08afb1a34e529808cbfd98260d39/users/22e3c936aae542d796147228db81b39f/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:30.020572 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc2658-c5d4-416f-a576-5c996b70d320 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:30.149868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7fe96-6e3f-451a-89c7-6979c727ae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:30.151154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7fe96-6e3f-451a-89c7-6979c727ae01 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.152238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7fe96-6e3f-451a-89c7-6979c727ae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.152956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7fe96-6e3f-451a-89c7-6979c727ae01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.154688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7fe96-6e3f-451a-89c7-6979c727ae01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:30.157942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7fe96-6e3f-451a-89c7-6979c727ae01 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:30.180548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1028/2068] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:30.187784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:30.212685 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28ef81-77d1-4352-8a68-dc02d4a8cf32 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:30.220575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:30.222207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.222690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.223187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] PATH_INFO: `/v3/projects/bb6d08afb1a34e529808cbfd98260d39/users/22e3c936aae542d796147228db81b39f/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.227026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6d08afb1a34e529808cbfd98260d39, user_id=22e3c936aae542d796147228db81b39f, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:30.233387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:30.252449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1041/2069] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:30 2022] PUT /identity/v3/projects/bb6d08afb1a34e529808cbfd98260d39/users/22e3c936aae542d796147228db81b39f/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:30.270945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.271448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.271939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.334422 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] MFA Rules not processed for user `22e3c936aae542d796147228db81b39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:30.337209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:30.354730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:30.462833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:30.466400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:30.496475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1029/2070] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:30.518069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:30.520019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:30.612853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da578b6c-d002-4fca-a3d2-00034f7763be None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:30.618879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:30.619651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.620213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.620414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.625143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bb3d3-7a30-4eb1-baa8-e3a8709e0b15 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:30.632241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:30.633101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:30.634593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.635100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.635614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] PATH_INFO: `/v3/projects/eb9b76b44b754456acb2c223a76d59f4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.636344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:30.638039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9b76b44b754456acb2c223a76d59f4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:30.641871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:30.650713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1030/2071] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:30.734224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9b76b44b754456acb2c223a76d59f4', 'request_id': 'req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:30.746845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9b76b44b754456acb2c223a76d59f4', 'request_id': 'req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:30.761121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9b76b44b754456acb2c223a76d59f4', 'request_id': 'req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:30.795340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] The token cache is being invalidate because project eb9b76b44b754456acb2c223a76d59f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:30.796345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:30.797960 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:30.804357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1042/2072] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:30 2022] DELETE /identity/v3/projects/eb9b76b44b754456acb2c223a76d59f4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:30.810022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:30.830976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486b1e4-ba06-40cd-8c12-af0c63fe227c service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:30.869934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:30.872940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:30.944312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:30.949382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:30.950307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.950592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.950878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] PATH_INFO: `/v3/users/c012e30895d34dc3b573ef7ab3c5b2e7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.954952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c012e30895d34dc3b573ef7ab3c5b2e7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:30.961238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:30.980457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061c4c5-7e74-46e1-8f62-4772c01f2cf8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:30.986372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:30.987665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:30.988134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:30.988658 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] PATH_INFO: `/v3/users/74b4d88cd8be4d0f816143cc6f8fce0a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:30.993818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b4d88cd8be4d0f816143cc6f8fce0a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:31.004310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:31.082505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c012e30895d34dc3b573ef7ab3c5b2e7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.087844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:31.109227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c012e30895d34dc3b573ef7ab3c5b2e7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.124918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c012e30895d34dc3b573ef7ab3c5b2e7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.127199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c012e30895d34dc3b573ef7ab3c5b2e7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.128347 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3e43fa3-b960-45fe-bb16-45b97616eec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c012e30895d34dc3b573ef7ab3c5b2e7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.141364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b4d88cd8be4d0f816143cc6f8fce0a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.161381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1031/2073] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:30 2022] DELETE /identity/v3/users/c012e30895d34dc3b573ef7ab3c5b2e7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:31.173715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b4d88cd8be4d0f816143cc6f8fce0a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.192499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b4d88cd8be4d0f816143cc6f8fce0a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.212822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b4d88cd8be4d0f816143cc6f8fce0a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.214872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b4d88cd8be4d0f816143cc6f8fce0a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.225351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1043/2074] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:30 2022] DELETE /identity/v3/users/74b4d88cd8be4d0f816143cc6f8fce0a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:31.231920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:31.234149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:31.235037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:31.272826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:31.275174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:31.386761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1032/2075] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:31.407987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:31.417610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:31.554260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33da92d-4237-4d98-bf7b-4c4c5731bc59 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:31.561766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b1a5-9bb0-4ce4-8a64-9d8a17ecb466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:31.563555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b1a5-9bb0-4ce4-8a64-9d8a17ecb466 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:31.564324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b1a5-9bb0-4ce4-8a64-9d8a17ecb466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:31.564997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b1a5-9bb0-4ce4-8a64-9d8a17ecb466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:31.593470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257b1a5-9bb0-4ce4-8a64-9d8a17ecb466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:31.596225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257b1a5-9bb0-4ce4-8a64-9d8a17ecb466 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:31.618492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1044/2076] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:31.651094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:31.652740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:31.666667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa532164-a55c-4583-9fb2-8f1a99eac2f3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:31.673295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:31.674614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:31.675022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:31.676064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] PATH_INFO: `/v3/projects/31d5bda55e6044ddab94ce68368b9200` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:31.678220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d5bda55e6044ddab94ce68368b9200)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:31.680592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:31.781895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d5bda55e6044ddab94ce68368b9200', 'request_id': 'req-ae6008f8-9646-4e57-9ebe-14308ff73c05'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.801210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d5bda55e6044ddab94ce68368b9200', 'request_id': 'req-ae6008f8-9646-4e57-9ebe-14308ff73c05'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.825228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d5bda55e6044ddab94ce68368b9200', 'request_id': 'req-ae6008f8-9646-4e57-9ebe-14308ff73c05'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.852722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] The token cache is being invalidate because project 31d5bda55e6044ddab94ce68368b9200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:31.853363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.855131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae6008f8-9646-4e57-9ebe-14308ff73c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:31.858052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1033/2077] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:31 2022] DELETE /identity/v3/projects/31d5bda55e6044ddab94ce68368b9200 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:31.864254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:31.864760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:31.865259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:31.904485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:31.904485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:32.047534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1045/2078] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:32.057445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.057955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.058453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.098531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:32.100695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:32.131514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1034/2079] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:32.136916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.185847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633c09-01de-40d0-8bec-bfd2751c5724 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.196217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.196947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.197552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.198470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.202166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:32.202846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.205367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.211453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:32.230237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.238874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1046/2080] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:32.242197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.246323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4106a-6180-42b3-9220-6739100da01f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.248991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:32.254434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.255598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.256094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.256594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.257540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.262610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.278308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c26f22-e978-4b5d-bd63-320705e1e3b4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.297276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.298522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.299002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.299520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] PATH_INFO: `/v3/projects/784bedc01b834c2980092e8d463c12e8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.310282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=784bedc01b834c2980092e8d463c12e8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.313192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.358326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1035/2081] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:32 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:32.360954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.402573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:32.412006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784bedc01b834c2980092e8d463c12e8', 'request_id': 'req-5f5d3248-bf90-4357-87f0-7063d2b79afc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.432427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784bedc01b834c2980092e8d463c12e8', 'request_id': 'req-5f5d3248-bf90-4357-87f0-7063d2b79afc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.448633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784bedc01b834c2980092e8d463c12e8', 'request_id': 'req-5f5d3248-bf90-4357-87f0-7063d2b79afc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.477050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] The token cache is being invalidate because project 784bedc01b834c2980092e8d463c12e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:32.478127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.479463 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.482606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70407b90-dc73-44c9-bf56-9a1a5af0613d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.488299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1047/2082] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:32 2022] DELETE /identity/v3/projects/784bedc01b834c2980092e8d463c12e8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:32.490026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.494104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:32.581529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.582450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.583523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.583523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.584701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.588769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.646791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1036/2083] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:32 2022] POST /identity/v3/users => generated 285 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:32.648019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.651257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:32.654265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d3248-bf90-4357-87f0-7063d2b79afc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.666593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.668024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c1e01-1249-4cb6-bfb5-f08fc34f631f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.668939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.669738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.670690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] PATH_INFO: `/v3/users/22e3c936aae542d796147228db81b39f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.673062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e3c936aae542d796147228db81b39f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.675091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.676684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.676982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.677888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.678652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.680856 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.684578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.707715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1037/2084] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:32.714265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.734828 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed4ea5-f2ed-4287-b082-e6a9b42566b8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.746146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.747374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.747878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.748383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.749972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.753205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.760474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e3c936aae542d796147228db81b39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.782581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1038/2085] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:32.785081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:32.792976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e3c936aae542d796147228db81b39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.807327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a2eb0-2548-4a68-88b6-0d311a4d2879 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:32.811823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e3c936aae542d796147228db81b39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.815980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:32.817787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.818179 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.818696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] PATH_INFO: `/v3/projects/977274af64b2438db7b87ef9dbf18ea5/users/cef145adfa384ef7878ce4d612996e70/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.823504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e3c936aae542d796147228db81b39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.824467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35c02d32-6afd-40a8-81b5-4e4e3e7511b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e3c936aae542d796147228db81b39f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:32.829984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:32.839220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1048/2086] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:32 2022] DELETE /identity/v3/users/22e3c936aae542d796147228db81b39f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:32.865824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=977274af64b2438db7b87ef9dbf18ea5, user_id=cef145adfa384ef7878ce4d612996e70, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:32.871177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:32.901478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1039/2087] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:32 2022] PUT /identity/v3/projects/977274af64b2438db7b87ef9dbf18ea5/users/cef145adfa384ef7878ce4d612996e70/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:32.920770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:32.921359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:32.922071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:32.978743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] MFA Rules not processed for user `cef145adfa384ef7878ce4d612996e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:32.981148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:33.154788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1049/2088] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:33.176936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:33.179953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:33.366690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5b820-9568-499a-adee-df3d81009f5a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:33.374155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b0f0a-1289-4e83-ad36-0a5b4fb68a31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:33.375543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0f0a-1289-4e83-ad36-0a5b4fb68a31 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:33.376015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0f0a-1289-4e83-ad36-0a5b4fb68a31 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:33.376629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b0f0a-1289-4e83-ad36-0a5b4fb68a31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:33.390393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:33.398446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:33.400286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b0f0a-1289-4e83-ad36-0a5b4fb68a31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:33.402003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:33.405596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b0f0a-1289-4e83-ad36-0a5b4fb68a31 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:33.430291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1040/2089] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:33.623825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320ac02-0f6f-48d0-9408-1ca32def989a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:33.632142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:33.634899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:33.635395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:33.635890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] PATH_INFO: `/v3/projects/bb6d08afb1a34e529808cbfd98260d39` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:33.638355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6d08afb1a34e529808cbfd98260d39)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:33.642340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:33.822780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6d08afb1a34e529808cbfd98260d39', 'request_id': 'req-d4702f6e-e478-4263-98a8-385c8fc24672'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:33.833388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6d08afb1a34e529808cbfd98260d39', 'request_id': 'req-d4702f6e-e478-4263-98a8-385c8fc24672'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:33.846930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6d08afb1a34e529808cbfd98260d39', 'request_id': 'req-d4702f6e-e478-4263-98a8-385c8fc24672'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:33.875782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] The token cache is being invalidate because project bb6d08afb1a34e529808cbfd98260d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:33.876360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:33.877619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:33.880760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1050/2090] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:33 2022] DELETE /identity/v3/projects/bb6d08afb1a34e529808cbfd98260d39 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:33.888551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb403-80d5-4d4d-8a42-fc29329415a0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:33.889100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb403-80d5-4d4d-8a42-fc29329415a0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:33.889621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb403-80d5-4d4d-8a42-fc29329415a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:33.941913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a6adb403-80d5-4d4d-8a42-fc29329415a0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:33.944963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a6adb403-80d5-4d4d-8a42-fc29329415a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:34.010232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.012730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:34.050211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:34.112616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1041/2091] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:34.126607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.127336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.127808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.164076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:34.166690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:34.197321 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1042/2092] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:34.201138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.214213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4702f6e-e478-4263-98a8-385c8fc24672 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:34.221390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:34.223629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16e785-0faf-4229-b2e5-7e7b9ac787dd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:34.225489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.226334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.228367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] PATH_INFO: `/v3/users/621d0d2532fd4e3c96012096f5a90f47` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.241663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=621d0d2532fd4e3c96012096f5a90f47)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:34.248328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:34.249366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:34.253044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.253441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.254723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.256687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:34.258371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:34.262718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:34.308536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1043/2093] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:34.315577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.330874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43a7f-1a2c-4f33-b707-e7d57f34dc15 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:34.348531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:34.349191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.349480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.349784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.350508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:34.352559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:34.397552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621d0d2532fd4e3c96012096f5a90f47'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:34.432107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1044/2094] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:34 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:34.450677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.460751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621d0d2532fd4e3c96012096f5a90f47'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:34.479251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621d0d2532fd4e3c96012096f5a90f47'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:34.500582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621d0d2532fd4e3c96012096f5a90f47'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:34.503567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-72e302a0-e8fe-481a-9f52-f63f6f6ede7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621d0d2532fd4e3c96012096f5a90f47'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:34.535957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1051/2095] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:34 2022] DELETE /identity/v3/users/621d0d2532fd4e3c96012096f5a90f47 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:34.548242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.548918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.549482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.600818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:34.603950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:34.640742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb054d-976e-44dd-8394-5e8cc44f8076 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:34.831611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1052/2096] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:34.831884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:34.831884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.831884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.831884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.833141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:34.833255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:34.841014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.843929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:34.856372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:34.884169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1045/2097] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:34 2022] POST /identity/v3/users => generated 290 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:34.890391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.893336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:34.921840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55064-25bf-4e00-9e7d-24785fbe2ada admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:34.933383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:34.934335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:34.935573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:34.937601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:34.940941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:34.941460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:34.951674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:34.971350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1046/2098] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:34.979806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:34.996591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd05d0-f525-420f-89a7-42e5a4e16555 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:35.004488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4c4b2-8f68-40fe-ad05-3b854fa8475b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:35.005654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:35.007142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:35.007666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:35.008446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:35.019975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a71d8-7be9-404b-aff4-bc6919d49315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:35.021965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a71d8-7be9-404b-aff4-bc6919d49315 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:35.022824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a71d8-7be9-404b-aff4-bc6919d49315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:35.023411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a71d8-7be9-404b-aff4-bc6919d49315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:35.029259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:35.030220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a71d8-7be9-404b-aff4-bc6919d49315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:35.032545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a71d8-7be9-404b-aff4-bc6919d49315 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:35.032704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:35.054445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1053/2099] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:35.066584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1047/2100] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:35.069978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:35.084523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283d911-594e-4f58-8055-fe01a817d371 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:35.093545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:35.094673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:35.095123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:35.095666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] PATH_INFO: `/v3/projects/f7f441cebef04fbd9b371960096095e6/users/333b829eb30d43468e53acc4d3569f55/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:35.118716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f441cebef04fbd9b371960096095e6, user_id=333b829eb30d43468e53acc4d3569f55, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:35.122276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:35.146089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1054/2101] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:35 2022] PUT /identity/v3/projects/f7f441cebef04fbd9b371960096095e6/users/333b829eb30d43468e53acc4d3569f55/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:35.166628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:35.168995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:35.173623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:35.256779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] MFA Rules not processed for user `333b829eb30d43468e53acc4d3569f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:35.258813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:35.418288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1048/2102] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:35.423279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:35.641325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8871a7d-6031-4193-968d-53438ea96fa2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:35.683293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:35.684578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:35.685127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:35.685599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:35.687015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:35.722230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:35.866758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1055/2103] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:35 2022] POST /identity/v3/projects => generated 377 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:35.872257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:35.939497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b069e5-08ea-41b3-bc99-9f2e1a95d737 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.016287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:36.018911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:36.125165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.126682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.127029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.127318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.128089 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.130331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.184438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1049/2104] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:35 2022] POST /identity/v3/users => generated 289 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:36.195108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:36.217732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f2c95-d469-4708-9e1a-d0f7b4c1921f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.227645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.228601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.228885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.229295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] PATH_INFO: `/v3/projects/894ea62d53ec4ecfade892d6014f7012` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.230935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=894ea62d53ec4ecfade892d6014f7012)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.233576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.237676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92d2c-f06f-4f53-a903-f5594524e64b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.257843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.259098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.259388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.259667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.260482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.262217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.287647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1050/2105] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:36.298175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:36.336378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23497-0f67-4bf5-a215-ee243a248ead admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.353325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.355970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.355970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.355970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.357329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.371275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.407385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:36.414295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894ea62d53ec4ecfade892d6014f7012', 'request_id': 'req-758c0a54-591d-44be-a793-2203dcb579c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.417903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1051/2106] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:36.419474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:36.432978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f3157-6b35-41db-a541-91453ad59480 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.435493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894ea62d53ec4ecfade892d6014f7012', 'request_id': 'req-758c0a54-591d-44be-a793-2203dcb579c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.454567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894ea62d53ec4ecfade892d6014f7012', 'request_id': 'req-758c0a54-591d-44be-a793-2203dcb579c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.468663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:36.508014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] The token cache is being invalidate because project 894ea62d53ec4ecfade892d6014f7012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:36.508014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.508014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.516045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1056/2107] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:36 2022] DELETE /identity/v3/projects/894ea62d53ec4ecfade892d6014f7012 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:36.523168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:36.526352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:36.572895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4f64c-0e40-4f44-8223-6bc0f0a3a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.573894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4f64c-0e40-4f44-8223-6bc0f0a3a579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.574378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4f64c-0e40-4f44-8223-6bc0f0a3a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.575307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4f64c-0e40-4f44-8223-6bc0f0a3a579 admin admin] PATH_INFO: `/v3/projects/4f12ee189cc0473c8d19c96145aacecd/users/5ac2f6f1f4cf4b91b838fca9a5c6d032/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.607988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4f64c-0e40-4f44-8223-6bc0f0a3a579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f12ee189cc0473c8d19c96145aacecd, user_id=5ac2f6f1f4cf4b91b838fca9a5c6d032, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.614150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4f64c-0e40-4f44-8223-6bc0f0a3a579 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.638184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c0a54-591d-44be-a793-2203dcb579c8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.645350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1052/2108] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:36 2022] PUT /identity/v3/projects/4f12ee189cc0473c8d19c96145aacecd/users/5ac2f6f1f4cf4b91b838fca9a5c6d032/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:36.662723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.663218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.664011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.746723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] MFA Rules not processed for user `5ac2f6f1f4cf4b91b838fca9a5c6d032`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:36.749137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:36.761124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.763125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.763684 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.764275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] PATH_INFO: `/v3/users/be764cc4456f40359fbd20183c2f1fc8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.768976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be764cc4456f40359fbd20183c2f1fc8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.774194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.873286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1053/2109] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:36.879644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:36.882236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:36.891278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be764cc4456f40359fbd20183c2f1fc8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.899775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7918626-8abc-46c9-91f8-fb3576e3dc97 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:36.910016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:36.912882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be764cc4456f40359fbd20183c2f1fc8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.913608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:36.914472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:36.915285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:36.916980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:36.923705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:36.934149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be764cc4456f40359fbd20183c2f1fc8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.954513 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be764cc4456f40359fbd20183c2f1fc8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.955966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be764cc4456f40359fbd20183c2f1fc8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:36.959284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:36.974040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1057/2110] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:36 2022] DELETE /identity/v3/users/be764cc4456f40359fbd20183c2f1fc8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:37.000450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1054/2111] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:36 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:37.005912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:37.008296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:37.151907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c18ed2-52c3-4d0a-aa76-038ecb53aab7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:37.250512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:37.251950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:37.252230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:37.252521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:37.253274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:37.261787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:37.358446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1058/2112] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:37 2022] POST /identity/v3/users => generated 295 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:37.366303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:37.426429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e358-64d2-4232-9ed7-4c0cfba1434e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:37.435511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:37.437453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:37.437832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:37.438234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:37.439152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:37.452460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:37.465725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1055/2113] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:37.472729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:37.489387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:37.496236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:37.529994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955bef-1490-408d-b89a-c964bcdd2fd8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:37.542417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:37.544552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:37.545076 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:37.545552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] PATH_INFO: `/v3/projects/fcf12ac9262a41bcb017aa53b154a6ef/users/e6215a260dba4800a0e62951a4c266a6/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:37.546528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4ee06-10b9-4953-b072-a593cf607bb0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:37.553462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:37.556764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:37.556764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:37.556764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] PATH_INFO: `/v3/projects/4d64125bbffe4d619cb26897cfbec7c6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:37.558077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d64125bbffe4d619cb26897cfbec7c6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:37.560266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:37.618478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf12ac9262a41bcb017aa53b154a6ef, user_id=e6215a260dba4800a0e62951a4c266a6, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:37.625427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:37.682797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1059/2114] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:37 2022] PUT /identity/v3/projects/fcf12ac9262a41bcb017aa53b154a6ef/users/e6215a260dba4800a0e62951a4c266a6/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:37.690768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:37.707800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d64125bbffe4d619cb26897cfbec7c6', 'request_id': 'req-2b665eea-b044-4966-97e7-bf8c03376858'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:37.724698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d64125bbffe4d619cb26897cfbec7c6', 'request_id': 'req-2b665eea-b044-4966-97e7-bf8c03376858'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:37.764360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d64125bbffe4d619cb26897cfbec7c6', 'request_id': 'req-2b665eea-b044-4966-97e7-bf8c03376858'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:37.768705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:37.813024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] The token cache is being invalidate because project 4d64125bbffe4d619cb26897cfbec7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:37.815482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:37.817493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b665eea-b044-4966-97e7-bf8c03376858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:37.821187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1056/2115] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:37 2022] DELETE /identity/v3/projects/4d64125bbffe4d619cb26897cfbec7c6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:37.836255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c82d9-419c-4f88-9a91-7cf1d51ac9d2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:37.837117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c82d9-419c-4f88-9a91-7cf1d51ac9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:37.837371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c82d9-419c-4f88-9a91-7cf1d51ac9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:37.880097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-da2c82d9-419c-4f88-9a91-7cf1d51ac9d2 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:37.881878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-da2c82d9-419c-4f88-9a91-7cf1d51ac9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:37.894176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31a13-2e08-42e6-9e81-156bdbec67c2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:37.948282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:38.062549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1057/2116] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:38.072273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.073383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.074023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.075501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.077090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.077875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.078497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.079194 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.079371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.115779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1060/2117] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:38.118282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.121842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:38.123688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:38.124986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:38.140136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1058/2118] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:38.141509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.158565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6823fa-8807-41ef-b674-f9754aa26e67 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.174559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957e3b0-2cc5-403a-bdf3-284b3010df99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.176283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957e3b0-2cc5-403a-bdf3-284b3010df99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.176836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957e3b0-2cc5-403a-bdf3-284b3010df99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.177333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957e3b0-2cc5-403a-bdf3-284b3010df99 admin admin] PATH_INFO: `/v3/projects/fcf12ac9262a41bcb017aa53b154a6ef/users/e6215a260dba4800a0e62951a4c266a6/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.177987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92808214-08bb-4fc1-acd0-5ce52b4cfa13 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.182111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957e3b0-2cc5-403a-bdf3-284b3010df99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf12ac9262a41bcb017aa53b154a6ef, user_id=e6215a260dba4800a0e62951a4c266a6, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.188796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957e3b0-2cc5-403a-bdf3-284b3010df99 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.194622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.197040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.197855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.198601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.200040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:38.200888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.204125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.211128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1061/2119] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:38 2022] PUT /identity/v3/projects/fcf12ac9262a41bcb017aa53b154a6ef/users/e6215a260dba4800a0e62951a4c266a6/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:38.232194 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1059/2120] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:38.234774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.235407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.235717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.240568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.272666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d386f-e700-42de-b35c-69bdc3faec8e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.316079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] MFA Rules not processed for user `e6215a260dba4800a0e62951a4c266a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:38.318303 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:38.388332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.389557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.390060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.390539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.391852 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.395682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.450487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1060/2121] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:38 2022] POST /identity/v3/projects => generated 407 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:38.456407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.457080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1062/2122] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:38.490670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.494069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:38.621548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:38.634273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c052e6-d769-4372-8d59-c3ff12d2ea5a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.642267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f7c1d-9072-4ac7-a00e-d578474f0fee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.647814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.648559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.648850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.649838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.651842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.653987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.704060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1061/2123] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:38 2022] POST /identity/v3/users => generated 304 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:38.705762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.725738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8207d2c-aa0a-4dbe-bba3-8e632fe002c5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.750434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.751744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.752313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.752925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.755162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.775538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.816685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1062/2124] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:38.818959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.822456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:38.843890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb799ce9-65f4-4277-9b5f-7fcaab241160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.846001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb799ce9-65f4-4277-9b5f-7fcaab241160 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.846439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb799ce9-65f4-4277-9b5f-7fcaab241160 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.847348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb799ce9-65f4-4277-9b5f-7fcaab241160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.858409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ea783-2dde-4322-9d00-773f95e9ea33 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:38.868700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:38.869385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:38.869844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:38.870392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:38.871628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:38.883242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:38.937865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1063/2125] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:38.940024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:38.988326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55af1e-d597-4f9d-9d69-2bc5cfee33c0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.009210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.011408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.014770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.015598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] PATH_INFO: `/v3/projects/17118e710bdc4116b5621d7333e69064/users/69121a15a0714ff9a700f0be2580cbe1/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.078442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb799ce9-65f4-4277-9b5f-7fcaab241160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.080277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb799ce9-65f4-4277-9b5f-7fcaab241160 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.085970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17118e710bdc4116b5621d7333e69064, user_id=69121a15a0714ff9a700f0be2580cbe1, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.101692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.121036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1063/2126] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:39.143084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1064/2127] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:38 2022] PUT /identity/v3/projects/17118e710bdc4116b5621d7333e69064/users/69121a15a0714ff9a700f0be2580cbe1/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:39.154196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.154762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.155277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.228252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] MFA Rules not processed for user `69121a15a0714ff9a700f0be2580cbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:39.229632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:39.256049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:39.258018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:39.397451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1064/2128] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:39.398744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:39.423010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaea53-43ea-4df2-9a50-45f4876b1b4d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.432691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.434174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.434460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.434739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.436922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:39.509732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72b5a2-78ff-4f95-bace-e0e567c2cf44 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.516821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.517694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.518056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.518401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.519189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.521891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.525095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.526564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.537446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1065/2129] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:39.538408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:39.540488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:39.557281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65ad49-d2a5-4e13-a6e9-94d5bd78b5b0 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.560113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1065/2130] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:39 2022] POST /identity/v3/projects => generated 409 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:39.565089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:39.567154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.568046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.568413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.568870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] PATH_INFO: `/v3/users/13e70b3794a445fc8deeb5f967eef673` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.572661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e70b3794a445fc8deeb5f967eef673)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.574891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.581050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:39.658744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e70b3794a445fc8deeb5f967eef673'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:39.660568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:39.684266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e524dc4-c364-4546-89d9-88b6122db7f9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.693427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.694531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.694843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e70b3794a445fc8deeb5f967eef673'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:39.695005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.695441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.697389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.700655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.709155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e70b3794a445fc8deeb5f967eef673'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:39.710883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e70b3794a445fc8deeb5f967eef673'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:39.712178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e70b3794a445fc8deeb5f967eef673'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:39.739676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1066/2131] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:39 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:39.741647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:39.745523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:39.759832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1066/2132] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:39 2022] DELETE /identity/v3/users/13e70b3794a445fc8deeb5f967eef673 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:39.769162 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16896f44-86c3-4a54-afae-328727cd777b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.892533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.894846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.895601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.896060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.897439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.900521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:39.923007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1067/2133] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:39.927854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:39.943835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446689d-2ed9-4031-aeff-84a2ad4c7ec4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:39.961357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:39.969813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:39.970126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:39.970493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] PATH_INFO: `/v3/projects/d5513d44eab34d35a626ff9e4cebaafe/users/0d477c6e7987472292bfc43b32ce507d/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:39.990632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5513d44eab34d35a626ff9e4cebaafe, user_id=0d477c6e7987472292bfc43b32ce507d, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:39.994205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.036808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1067/2134] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:39 2022] PUT /identity/v3/projects/d5513d44eab34d35a626ff9e4cebaafe/users/0d477c6e7987472292bfc43b32ce507d/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:40.038771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.063326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e1b31-d0d2-415d-bd62-5fb653ddfce7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.103714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.109814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:40.291542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6fd84-0348-488e-befb-d6dec1180713 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.301783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.302393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.302874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.303235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.303691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] PATH_INFO: `/v3/projects/d3c9ca821cd341a98f21ca15a0d6eb4c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.304476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.305017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.305535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.305928 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c9ca821cd341a98f21ca15a0d6eb4c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.307002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.309392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.311079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.334581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1068/2135] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:40.335859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.349132 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d97602-1da5-4cd4-83da-48397027f268 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.361762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249724c7-ce6e-404e-960e-fa834ed1ccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.363968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249724c7-ce6e-404e-960e-fa834ed1ccf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.363968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249724c7-ce6e-404e-960e-fa834ed1ccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.363968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249724c7-ce6e-404e-960e-fa834ed1ccf6 admin admin] PATH_INFO: `/v3/projects/d5513d44eab34d35a626ff9e4cebaafe/users/0d477c6e7987472292bfc43b32ce507d/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.366255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249724c7-ce6e-404e-960e-fa834ed1ccf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5513d44eab34d35a626ff9e4cebaafe, user_id=0d477c6e7987472292bfc43b32ce507d, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.369009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249724c7-ce6e-404e-960e-fa834ed1ccf6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.385093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1069/2136] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:40 2022] PUT /identity/v3/projects/d5513d44eab34d35a626ff9e4cebaafe/users/0d477c6e7987472292bfc43b32ce507d/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:40.407304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.407609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.407933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.418172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c9ca821cd341a98f21ca15a0d6eb4c', 'request_id': 'req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:40.435167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c9ca821cd341a98f21ca15a0d6eb4c', 'request_id': 'req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:40.447835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c9ca821cd341a98f21ca15a0d6eb4c', 'request_id': 'req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:40.465746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] The token cache is being invalidate because project d3c9ca821cd341a98f21ca15a0d6eb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:40.466028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:40.466612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] MFA Rules not processed for user `0d477c6e7987472292bfc43b32ce507d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:40.467615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-472c995f-a5e9-4ce8-b5c3-8706c4b0b036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:40.468688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:40.470280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1068/2137] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:40 2022] DELETE /identity/v3/projects/d3c9ca821cd341a98f21ca15a0d6eb4c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:40.478096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:40.490450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323c8a-aa5a-4c6d-b426-dc564490bb68 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.490722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323c8a-aa5a-4c6d-b426-dc564490bb68 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.491425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323c8a-aa5a-4c6d-b426-dc564490bb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.517734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d7323c8a-aa5a-4c6d-b426-dc564490bb68 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:40.519256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d7323c8a-aa5a-4c6d-b426-dc564490bb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:40.556697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:40.614029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1069/2138] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:40.625642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1070/2139] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:40.627393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.627755 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.628099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.643175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.645385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:40.670426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:40.672794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:40.687861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1070/2140] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:40.695161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.722550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae73c4-6daf-47d8-ab44-570d320d1c64 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.729141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43f47c-e863-4211-bc3f-2fc23ea69a21 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.729389 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.730125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.730615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.731111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.732229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:40.732784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.734414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.735350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.735688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.736051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.740022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.749701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.752029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.757238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1071/2141] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:40.762160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.770557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1071/2142] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:40 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:40.773161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f83a8-6e36-426a-a531-8a6a1be0fed2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.780926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.784684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.785250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.785672 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.786596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.789399 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.827016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1072/2143] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:40 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:40.831560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.843915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfacd2-869c-46c3-83ac-da671bc814b6 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.926488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.927343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.927731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.928120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.929038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.932871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.961259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1072/2144] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:40 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:40.965616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:40.976954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d223e-cf04-4b29-8e97-e0d37208ddb2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:40.982372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:40.983109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:40.983397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:40.983700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:40.984576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:40.986196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:40.997848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1073/2145] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:41.002087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.009008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.014005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea4291-9e91-438c-9dfa-fe474cb18d72 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.018734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.019403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.019683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.019957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.020816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.022359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.041367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1073/2146] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:41.046123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.063730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281823f-f8e2-4b0f-b57b-7409e51fb17d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.069793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e14b76-4772-4376-9fca-057143e50839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.071030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14b76-4772-4376-9fca-057143e50839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.071312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14b76-4772-4376-9fca-057143e50839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.071635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14b76-4772-4376-9fca-057143e50839 admin admin] PATH_INFO: `/v3/projects/fb2d2f33a2894f6dad3aa8b61a760572/users/06982aac5f814618ae31436c70bb6372/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.081070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cbcb7-8386-4efe-bbdd-ba7eac439a0d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.088064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.089103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.089584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.090211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.091517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e14b76-4772-4376-9fca-057143e50839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2d2f33a2894f6dad3aa8b61a760572, user_id=06982aac5f814618ae31436c70bb6372, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.092978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:41.096877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e14b76-4772-4376-9fca-057143e50839 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.110909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:41.117830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1074/2147] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:41 2022] PUT /identity/v3/projects/fb2d2f33a2894f6dad3aa8b61a760572/users/06982aac5f814618ae31436c70bb6372/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:41.135209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.135579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.135871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.177876 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] MFA Rules not processed for user `06982aac5f814618ae31436c70bb6372`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:41.180769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:41.215946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.234014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.250206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1074/2148] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:41.292514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1075/2149] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:41.296964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.314442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905f667-a58b-4c0f-9778-5ae3360da998 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.386650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:41.432085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.432923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.433556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.433787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.434445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.436693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.482616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.487006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:41.511175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1075/2150] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:41 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:41.517994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.525122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a2c78-38f1-4616-a17f-979ece732afc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.659093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fbf5e-9f02-499b-8ff5-918819d43ee4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.660725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.662379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.662379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.662752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] PATH_INFO: `/v3/users/fd368a758a324546af3bc79b1909917a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.668460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd368a758a324546af3bc79b1909917a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.668460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.683832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.684781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.685062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.685383 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.686099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.697123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.737739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1076/2151] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:41 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:41.741145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.778353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d326e79-8274-4658-bcf5-e53ac4511d5e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.802700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.813358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.813656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.816956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.816956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.817234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:41.831092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd368a758a324546af3bc79b1909917a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:41.855039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd368a758a324546af3bc79b1909917a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:41.859990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1077/2152] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:41.878249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:41.891551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd368a758a324546af3bc79b1909917a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:41.909129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd368a758a324546af3bc79b1909917a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:41.910190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97dafc83-34f3-4f01-92ea-2402ccdde575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd368a758a324546af3bc79b1909917a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:41.911185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b65192-1887-4ce2-b108-3a2558368ad8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:41.925023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:41.926330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.926940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.927903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] PATH_INFO: `/v3/projects/bf002167eff44b32ad480289326b3321/users/c0e85e81081747549b7a977f55ff6817/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.931819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1076/2153] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:41 2022] DELETE /identity/v3/users/fd368a758a324546af3bc79b1909917a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:41.936521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:41.937117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:41.937464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:41.953512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:41.965000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:41.965943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf002167eff44b32ad480289326b3321, user_id=c0e85e81081747549b7a977f55ff6817, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:41.968815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:41.971244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.013889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1078/2154] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:41 2022] PUT /identity/v3/projects/bf002167eff44b32ad480289326b3321/users/c0e85e81081747549b7a977f55ff6817/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:42.023400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:42.027694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:42.061388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1077/2155] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:42.076128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:42.078318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:42.159290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb5f19-5bf5-40e3-ac2a-883d27cc0050 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:42.165870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:42.166559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.166840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.167115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.168423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:42.171989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.175014 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00e175-aaab-4eeb-abef-91417866056d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:42.183831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1079/2156] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:42.193851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:42.213507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2670d-2242-4e52-a984-ad75d1999517 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:42.220861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30c7b4-bbfe-4d8e-947a-fcf9b4c6b0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:42.221779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30c7b4-bbfe-4d8e-947a-fcf9b4c6b0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.222055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30c7b4-bbfe-4d8e-947a-fcf9b4c6b0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.222337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30c7b4-bbfe-4d8e-947a-fcf9b4c6b0ff admin admin] PATH_INFO: `/v3/projects/bf002167eff44b32ad480289326b3321/users/c0e85e81081747549b7a977f55ff6817/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.224498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30c7b4-bbfe-4d8e-947a-fcf9b4c6b0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf002167eff44b32ad480289326b3321, user_id=c0e85e81081747549b7a977f55ff6817, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:42.227772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30c7b4-bbfe-4d8e-947a-fcf9b4c6b0ff admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.231300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:42.241170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:42.241843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.242126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.242402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.247536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1080/2157] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:42 2022] PUT /identity/v3/projects/bf002167eff44b32ad480289326b3321/users/c0e85e81081747549b7a977f55ff6817/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:42.258480 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.258777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.259068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.261252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:42.263768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.276064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1078/2158] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:42.290640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] MFA Rules not processed for user `c0e85e81081747549b7a977f55ff6817`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:42.291885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:42.403534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1081/2159] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:42.422112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:42.425941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:42.564145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccab3e2-8637-46f3-8d0b-efdf8861748a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:42.597338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:42.619902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:42.621504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:42.654882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:42.655567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.655933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.656286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.670970 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:42.672806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.684717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1079/2160] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:42.714501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:42.727602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3432ad-dab6-49a6-a7fd-5e020f4400fb service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:42.728375 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eccada-64d3-46be-9dbb-832670f0f12c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:42.733406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:42.733866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:42.735133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.735508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.735997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] PATH_INFO: `/v3/users/06982aac5f814618ae31436c70bb6372` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.737996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06982aac5f814618ae31436c70bb6372)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:42.739151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.739319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.739562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] PATH_INFO: `/v3/projects/b4c59bd900864f59bdc3cd109fee1ae8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.740470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.741017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c59bd900864f59bdc3cd109fee1ae8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:42.743259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:42.807481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c59bd900864f59bdc3cd109fee1ae8', 'request_id': 'req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.817123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c59bd900864f59bdc3cd109fee1ae8', 'request_id': 'req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.818091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06982aac5f814618ae31436c70bb6372'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.831834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c59bd900864f59bdc3cd109fee1ae8', 'request_id': 'req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.841695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06982aac5f814618ae31436c70bb6372'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.851076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] The token cache is being invalidate because project b4c59bd900864f59bdc3cd109fee1ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:42.851542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.853196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f236385-fc8f-4f9c-a2b9-ddaf3ce9d37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.855690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06982aac5f814618ae31436c70bb6372'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.855992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1082/2161] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:42 2022] DELETE /identity/v3/projects/b4c59bd900864f59bdc3cd109fee1ae8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:42.875870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06982aac5f814618ae31436c70bb6372'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.876524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06982aac5f814618ae31436c70bb6372'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:42.885105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1080/2162] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:42 2022] DELETE /identity/v3/users/06982aac5f814618ae31436c70bb6372 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:42.892473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:42.892945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:42.893360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:42.927725 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:42.929466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:42.941252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:43.024118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1083/2163] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:43.044641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:43.046755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:43.196872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef31977-f9c8-4f36-a9fd-f191c890526a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:43.201689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca38be-6f49-437a-acac-1b53fd47054b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:43.202366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca38be-6f49-437a-acac-1b53fd47054b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:43.203080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca38be-6f49-437a-acac-1b53fd47054b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:43.203080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca38be-6f49-437a-acac-1b53fd47054b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:43.215550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca38be-6f49-437a-acac-1b53fd47054b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:43.217443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca38be-6f49-437a-acac-1b53fd47054b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:43.245817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1081/2164] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:43.284090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:43.286683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:43.320591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf11657-6e6f-4048-b8b0-b7b75697525f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:43.349556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:43.351475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:43.352081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:43.352763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] PATH_INFO: `/v3/users/333b829eb30d43468e53acc4d3569f55` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:43.355330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] RBAC: Authorizing `identity:delete_user(user_id=333b829eb30d43468e53acc4d3569f55)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:43.359556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:43.459839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333b829eb30d43468e53acc4d3569f55'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.480927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333b829eb30d43468e53acc4d3569f55'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.499048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333b829eb30d43468e53acc4d3569f55'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.499899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333b829eb30d43468e53acc4d3569f55'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.500772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333b829eb30d43468e53acc4d3569f55'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.528868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1084/2165] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:43 2022] DELETE /identity/v3/users/333b829eb30d43468e53acc4d3569f55 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:43.535799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:43.536296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:43.536883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:43.575719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:43.580747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:43.643806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:43.681686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1082/2166] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:43.692786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:43.694446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:43.755697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:43.758149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:43.785973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5f1e0-78ab-46bc-b883-889dc587f29c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:43.795293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:43.796826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:43.797308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:43.797744 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] PATH_INFO: `/v3/projects/fb2d2f33a2894f6dad3aa8b61a760572` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:43.798387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0d92f-33e2-4273-874c-a31faef2965d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:43.799835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2d2f33a2894f6dad3aa8b61a760572)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:43.803639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:43.805748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:43.807247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:43.807881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:43.808960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:43.830535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:43.834034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:43.846022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1085/2167] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:43.894983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2d2f33a2894f6dad3aa8b61a760572', 'request_id': 'req-f3a66957-0879-45c0-8d37-ddf485f4f6ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.910423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2d2f33a2894f6dad3aa8b61a760572', 'request_id': 'req-f3a66957-0879-45c0-8d37-ddf485f4f6ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.925488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2d2f33a2894f6dad3aa8b61a760572', 'request_id': 'req-f3a66957-0879-45c0-8d37-ddf485f4f6ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.946668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] The token cache is being invalidate because project fb2d2f33a2894f6dad3aa8b61a760572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:43.947232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.948621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:43.951595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1083/2168] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:43 2022] DELETE /identity/v3/projects/fb2d2f33a2894f6dad3aa8b61a760572 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:43.958831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:43.978580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289f281-9236-4c34-9e0f-5a95af2819fb service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:44.007986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:44.050136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:44.051081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:44.051349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:44.051627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] PATH_INFO: `/v3/users/c0e85e81081747549b7a977f55ff6817` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:44.053108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e85e81081747549b7a977f55ff6817)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:44.055850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:44.125754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e85e81081747549b7a977f55ff6817'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.152434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e85e81081747549b7a977f55ff6817'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.166039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e85e81081747549b7a977f55ff6817'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.167296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e85e81081747549b7a977f55ff6817'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.168479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e85e81081747549b7a977f55ff6817'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.192020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1086/2169] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:43 2022] DELETE /identity/v3/users/c0e85e81081747549b7a977f55ff6817 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:44.276501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:44.278983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:44.311514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a66957-0879-45c0-8d37-ddf485f4f6ce admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:44.457236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:44.458301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:44.458679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:44.459054 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] PATH_INFO: `/v3/projects/f7f441cebef04fbd9b371960096095e6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:44.460525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f441cebef04fbd9b371960096095e6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:44.462883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:44.551495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f441cebef04fbd9b371960096095e6', 'request_id': 'req-cad0eca7-2162-40fa-913a-5b5009dcf512'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.560441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f441cebef04fbd9b371960096095e6', 'request_id': 'req-cad0eca7-2162-40fa-913a-5b5009dcf512'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.570378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f441cebef04fbd9b371960096095e6', 'request_id': 'req-cad0eca7-2162-40fa-913a-5b5009dcf512'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.577130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:44.587311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] The token cache is being invalidate because project f7f441cebef04fbd9b371960096095e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:44.588111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.590189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.592048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1084/2170] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:44 2022] DELETE /identity/v3/projects/f7f441cebef04fbd9b371960096095e6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:44.596937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:44.598714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:44.619829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718466b-3d8b-41e8-9547-1646bf9f1d9e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:44.699041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:44.703567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:44.703567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:44.703567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] PATH_INFO: `/v3/users/5ac2f6f1f4cf4b91b838fca9a5c6d032` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:44.708658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac2f6f1f4cf4b91b838fca9a5c6d032)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:44.715990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:44.772731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:44.779676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:44.833654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0eca7-2162-40fa-913a-5b5009dcf512 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:44.834638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac2f6f1f4cf4b91b838fca9a5c6d032'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.837174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:44.843332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:44.844583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:44.845099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:44.845530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] PATH_INFO: `/v3/projects/bf002167eff44b32ad480289326b3321` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:44.847755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf002167eff44b32ad480289326b3321)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:44.850107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:44.858573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac2f6f1f4cf4b91b838fca9a5c6d032'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.879798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac2f6f1f4cf4b91b838fca9a5c6d032'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.880779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac2f6f1f4cf4b91b838fca9a5c6d032'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.882900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-13dad7da-79ec-411c-86e9-5e60ba2f3652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac2f6f1f4cf4b91b838fca9a5c6d032'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.905996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1087/2171] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:44 2022] DELETE /identity/v3/users/5ac2f6f1f4cf4b91b838fca9a5c6d032 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:44.926466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf002167eff44b32ad480289326b3321', 'request_id': 'req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.934782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf002167eff44b32ad480289326b3321', 'request_id': 'req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.951977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf002167eff44b32ad480289326b3321', 'request_id': 'req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.971001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] The token cache is being invalidate because project bf002167eff44b32ad480289326b3321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:44.971331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.972036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04bb4f7e-88c4-4a28-8a50-8209ca5a9f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:44.973729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1085/2172] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:44 2022] DELETE /identity/v3/projects/bf002167eff44b32ad480289326b3321 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:44.982024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:44.982677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:44.983243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.035108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:45.036987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:45.137053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1088/2173] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:45.146000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.146512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.147030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.180984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:45.182460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:45.193252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1086/2174] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:45.198145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.208984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd418b-a773-4837-901f-37374bb687e4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.215186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.216023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.216380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.216766 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.217608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:45.218120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.220120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.247066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1089/2175] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:45.252160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.270818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915591de-926e-47fc-9286-3a935bf83db3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.279610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.280525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.281010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.281914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.282518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.285003 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.332998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1087/2176] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:45 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:45.338632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.358573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e23085-2426-4266-bac1-0debfd86debe admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.419666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.424267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:45.500573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.502151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.502744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.503241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.504894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.508004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.547788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1090/2177] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:45 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:45.549765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.574790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5bd66-57a6-4f22-bfaf-6cb9c5360e94 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.577514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbd9a8-2da3-4174-bef9-86127d61ef81 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.581647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.582787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.583348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.583825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.585252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.588566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.597183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.598277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.600088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.600446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] PATH_INFO: `/v3/projects/4f12ee189cc0473c8d19c96145aacecd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.603147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f12ee189cc0473c8d19c96145aacecd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.605222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.608219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1091/2178] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:45.612336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.630807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2249f48-d21a-4c40-a8be-9bbd80f70748 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.653193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.653193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.653193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.653193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.653633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.653633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.674258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1092/2179] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:45.680502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.686761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f12ee189cc0473c8d19c96145aacecd', 'request_id': 'req-03bec368-31da-4599-bacd-4d4d0a1f7eb9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:45.688982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:45.702313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f12ee189cc0473c8d19c96145aacecd', 'request_id': 'req-03bec368-31da-4599-bacd-4d4d0a1f7eb9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:45.703095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73badc68-e39c-49f5-8361-32b2130ce429 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.715862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f12ee189cc0473c8d19c96145aacecd', 'request_id': 'req-03bec368-31da-4599-bacd-4d4d0a1f7eb9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:45.730879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] The token cache is being invalidate because project 4f12ee189cc0473c8d19c96145aacecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:45.731323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:45.732647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:45.733311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:45.735870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1088/2180] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:45 2022] DELETE /identity/v3/projects/4f12ee189cc0473c8d19c96145aacecd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:45.741806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:45.743812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:45.818575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24bcf-95b6-4fa9-a5ca-29f8179e1a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.830512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24bcf-95b6-4fa9-a5ca-29f8179e1a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.830880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24bcf-95b6-4fa9-a5ca-29f8179e1a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.831403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24bcf-95b6-4fa9-a5ca-29f8179e1a92 admin admin] PATH_INFO: `/v3/projects/5135833399d643e4b290788d2f65c450/users/4a9fd1aaa09749aa8010579ec24995d4/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.851346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e24bcf-95b6-4fa9-a5ca-29f8179e1a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5135833399d643e4b290788d2f65c450, user_id=4a9fd1aaa09749aa8010579ec24995d4, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.854228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e24bcf-95b6-4fa9-a5ca-29f8179e1a92 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:45.866620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bec368-31da-4599-bacd-4d4d0a1f7eb9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:45.877912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1093/2181] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:45 2022] PUT /identity/v3/projects/5135833399d643e4b290788d2f65c450/users/4a9fd1aaa09749aa8010579ec24995d4/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:45.897608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.898217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.898843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.935593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:45.937245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:45.937707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:45.938155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] PATH_INFO: `/v3/users/e6215a260dba4800a0e62951a4c266a6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:45.939442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] MFA Rules not processed for user `4a9fd1aaa09749aa8010579ec24995d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:45.940285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6215a260dba4800a0e62951a4c266a6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:45.941424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:45.944068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.021681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6215a260dba4800a0e62951a4c266a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.035172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1094/2182] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:46.042147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.044094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.044840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6215a260dba4800a0e62951a4c266a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.058394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6215a260dba4800a0e62951a4c266a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.058683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7213346-8da0-49d7-8fd4-7684b53715b1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.063335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.064025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.064315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.064632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.065368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.067385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.072210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6215a260dba4800a0e62951a4c266a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.073677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6215a260dba4800a0e62951a4c266a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.082566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1089/2183] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:45 2022] DELETE /identity/v3/users/e6215a260dba4800a0e62951a4c266a6 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:46.116437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.118328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1095/2184] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:46 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:46.119840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.141916 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:46.249902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f81e-e717-4270-adb7-7097a4d98640 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.328200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.329080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.329456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.329825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.330682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.332838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.364592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1090/2185] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:46 2022] POST /identity/v3/users => generated 299 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:46.377819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.390829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d0eab-394f-496f-9475-eb64bd96918e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.396325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.397074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.397359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.397646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.398407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.400751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.415526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1096/2186] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:46.423084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.440765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bdf9f-ef48-412a-8c19-78f95381718c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.447671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.448734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.449042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.449294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] PATH_INFO: `/v3/projects/df65307a20694eb18dff46f7c64a6c91/users/13f1ddb0564446ab82ba111033446334/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.469775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df65307a20694eb18dff46f7c64a6c91, user_id=13f1ddb0564446ab82ba111033446334, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.472462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.472737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.474971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.491634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1091/2187] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:46 2022] PUT /identity/v3/projects/df65307a20694eb18dff46f7c64a6c91/users/13f1ddb0564446ab82ba111033446334/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:46.498620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.507129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4956aa-2ff3-4dd3-8be6-dee5882e3e77 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.603563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecc7f7-6e2c-4137-a262-54fa6419a3a8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.607889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.608830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.609810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.609988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.610163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] PATH_INFO: `/v3/projects/fcf12ac9262a41bcb017aa53b154a6ef` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.610320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.610320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.610620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.610817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf12ac9262a41bcb017aa53b154a6ef)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.611196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.612914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.613120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.615173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:46.626018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1092/2188] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:46.630852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.642481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb40d7-ede0-450c-b28e-961b29e88b2e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.648714 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffd3b2-0e83-4c53-b328-57cf2601ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.650418 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd3b2-0e83-4c53-b328-57cf2601ca57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.650782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd3b2-0e83-4c53-b328-57cf2601ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.651138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd3b2-0e83-4c53-b328-57cf2601ca57 admin admin] PATH_INFO: `/v3/projects/df65307a20694eb18dff46f7c64a6c91/users/13f1ddb0564446ab82ba111033446334/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.653551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffd3b2-0e83-4c53-b328-57cf2601ca57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df65307a20694eb18dff46f7c64a6c91, user_id=13f1ddb0564446ab82ba111033446334, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.656590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffd3b2-0e83-4c53-b328-57cf2601ca57 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.666248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:46.673367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1093/2189] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:46 2022] PUT /identity/v3/projects/df65307a20694eb18dff46f7c64a6c91/users/13f1ddb0564446ab82ba111033446334/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:46.688001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.699217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf12ac9262a41bcb017aa53b154a6ef', 'request_id': 'req-158fdd23-51f0-4053-be03-3824da168a5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.700774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.701446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.713225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf12ac9262a41bcb017aa53b154a6ef', 'request_id': 'req-158fdd23-51f0-4053-be03-3824da168a5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.727922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf12ac9262a41bcb017aa53b154a6ef', 'request_id': 'req-158fdd23-51f0-4053-be03-3824da168a5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.742207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] The token cache is being invalidate because project fcf12ac9262a41bcb017aa53b154a6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:46.743403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.744548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-158fdd23-51f0-4053-be03-3824da168a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:46.747252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1097/2190] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:46 2022] DELETE /identity/v3/projects/fcf12ac9262a41bcb017aa53b154a6ef => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:46.754257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f09dd2-e024-4666-a18c-15357640aee0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.754767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f09dd2-e024-4666-a18c-15357640aee0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.754953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] MFA Rules not processed for user `13f1ddb0564446ab82ba111033446334`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:46.755263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f09dd2-e024-4666-a18c-15357640aee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.757099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.780066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c5f09dd2-e024-4666-a18c-15357640aee0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:46.783543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c5f09dd2-e024-4666-a18c-15357640aee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.858787 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1094/2191] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:46.876128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.879391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.913981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1098/2192] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:46.916522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.916901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.917090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.944718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:46.944718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:46.953679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1099/2193] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:46.954583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a1bc-9552-489d-8a4d-9308d05f4d44 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.958360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:46.962275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.963644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.964274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.964966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.968675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7cd23-16dc-4798-99f5-087aef0251f3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:46.973508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:46.974214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:46.974497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:46.974773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:46.975535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:46.975846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.977711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:46.984773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:46.988026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.000690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1100/2194] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:47.003766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1095/2195] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:47.006074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.021084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608340-8d3b-4ef7-94ab-651af1846082 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.026945 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.028080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.028371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.028854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.029392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.032830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.074542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1101/2196] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:47 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:47.082679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.108545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5e7ad-e49e-45ca-9937-8dedb17e44c9 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.113721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.116055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:47.222735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.223435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.223721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.223992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.224748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.227594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.236380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:47.282256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1096/2197] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:47 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:47.283282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.291582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701de1a-48f3-4f3b-92fa-741f52c52fcd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.295199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad4fd0-5bfa-491d-ac61-4c64586eb771 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.300113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.300836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.301124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.301380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.301512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.302175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.303343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.303824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.304083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.304737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] PATH_INFO: `/v3/users/4a9fd1aaa09749aa8010579ec24995d4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.307938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a9fd1aaa09749aa8010579ec24995d4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.311449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.323489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1097/2198] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:47.327780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.339497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b474b7-6b18-405d-94a4-9a462634937d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.345808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.346590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.346878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.349115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.349115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.350938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.353789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:47.384362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1098/2199] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:47.389550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.410884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f87f1-66c9-4907-aae7-533185470985 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.422381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a9fd1aaa09749aa8010579ec24995d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:47.426681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2c85d-ec27-460d-a922-c29088040572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.428577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2c85d-ec27-460d-a922-c29088040572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.429342 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2c85d-ec27-460d-a922-c29088040572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.430199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2c85d-ec27-460d-a922-c29088040572 admin admin] PATH_INFO: `/v3/projects/c29dae21c3444b21afab84a9df540725/users/44ab217336e64971a91f4cd19f92eac7/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.458677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2c85d-ec27-460d-a922-c29088040572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29dae21c3444b21afab84a9df540725, user_id=44ab217336e64971a91f4cd19f92eac7, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.458965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a9fd1aaa09749aa8010579ec24995d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:47.462557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2c85d-ec27-460d-a922-c29088040572 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.482630 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a9fd1aaa09749aa8010579ec24995d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:47.483524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a9fd1aaa09749aa8010579ec24995d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:47.484328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fb8b689-f00c-4302-9379-7dfff27aca21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a9fd1aaa09749aa8010579ec24995d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:47.502068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1102/2200] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:47 2022] DELETE /identity/v3/users/4a9fd1aaa09749aa8010579ec24995d4 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:47.506863 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1099/2201] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:47 2022] PUT /identity/v3/projects/c29dae21c3444b21afab84a9df540725/users/44ab217336e64971a91f4cd19f92eac7/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:47.509898 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.510128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.510489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.529371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.540432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.540432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.541034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:47.541034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:47.597616 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] MFA Rules not processed for user `44ab217336e64971a91f4cd19f92eac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:47.599660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:47.623225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1103/2202] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:47.641435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.643167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:47.708445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1100/2203] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:47.731258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda39e9-041c-4448-81d9-b77968bf7a73 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.735796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:47.738156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:47.744221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.745812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.746311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.747060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.765798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.768993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.787582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1104/2204] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:47.882612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db3d03-aab1-4648-9076-13dfab1fa46e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:47.888487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:47.889180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:47.889528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:47.889973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:47.923105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:47.924772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:47.947509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1101/2205] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:48.251381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:48.253682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:48.282065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a601d-e706-4c9c-ac9d-af7a7456d521 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:48.292998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:48.294944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:48.294944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:48.295361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] PATH_INFO: `/v3/projects/5135833399d643e4b290788d2f65c450` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:48.300738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5135833399d643e4b290788d2f65c450)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:48.304154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:48.323535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:48.415746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5135833399d643e4b290788d2f65c450', 'request_id': 'req-18b56f71-8424-4666-a969-0c55da9f0ea7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.423753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5135833399d643e4b290788d2f65c450', 'request_id': 'req-18b56f71-8424-4666-a969-0c55da9f0ea7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.435351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5135833399d643e4b290788d2f65c450', 'request_id': 'req-18b56f71-8424-4666-a969-0c55da9f0ea7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.451608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] The token cache is being invalidate because project 5135833399d643e4b290788d2f65c450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:48.452141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.452996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.459003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1105/2206] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:48 2022] DELETE /identity/v3/projects/5135833399d643e4b290788d2f65c450 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:48.460651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:48.478800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c087b6-023c-4881-b428-846178634932 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:48.583386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:48.584745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:48.585141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:48.585517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] PATH_INFO: `/v3/users/13f1ddb0564446ab82ba111033446334` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:48.587558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f1ddb0564446ab82ba111033446334)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:48.589626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:48.596777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:48.680915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f1ddb0564446ab82ba111033446334'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.712575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f1ddb0564446ab82ba111033446334'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.726798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f1ddb0564446ab82ba111033446334'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.738399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f1ddb0564446ab82ba111033446334'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.739159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9df46344-ac9f-409f-8861-faeb599c21e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f1ddb0564446ab82ba111033446334'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:48.750566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1102/2207] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:48 2022] DELETE /identity/v3/users/13f1ddb0564446ab82ba111033446334 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:49.112644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:49.115271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:49.137941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56f71-8424-4666-a969-0c55da9f0ea7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:49.223619 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:49.224693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.225099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.225472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] PATH_INFO: `/v3/projects/df65307a20694eb18dff46f7c64a6c91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.227348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df65307a20694eb18dff46f7c64a6c91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:49.229680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:49.294033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df65307a20694eb18dff46f7c64a6c91', 'request_id': 'req-e0f4d79b-4a10-44a9-b92f-5faec45e877a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:49.301058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df65307a20694eb18dff46f7c64a6c91', 'request_id': 'req-e0f4d79b-4a10-44a9-b92f-5faec45e877a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:49.310840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df65307a20694eb18dff46f7c64a6c91', 'request_id': 'req-e0f4d79b-4a10-44a9-b92f-5faec45e877a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:49.324298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] The token cache is being invalidate because project df65307a20694eb18dff46f7c64a6c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:49.324611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:49.325292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e0f4d79b-4a10-44a9-b92f-5faec45e877a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:49.326908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1106/2208] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:49 2022] DELETE /identity/v3/projects/df65307a20694eb18dff46f7c64a6c91 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:49.332302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.332726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.333095 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.362909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:49.365103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:49.481919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1103/2209] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:49.490574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.491084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.491578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.527285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:49.531396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:49.546005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1107/2210] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:49.552513 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:49.568414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c135-cbd0-4af7-8f99-685a948ee06d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:49.580238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:49.581268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.581678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.582126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.583177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:49.583692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:49.585997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:49.620372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1104/2211] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:02:49.627500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:49.641461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d1b97-bd0a-48a4-8a6d-351036cff428 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:49.647574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:49.648719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.649174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.649640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.651056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:49.653349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:49.676208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:49.697688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1108/2212] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:49 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:49.704877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:49.723741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648517-93c0-4c3e-a687-fd36fe6c14bd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:49.848759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:49.850788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.851367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.853147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.856136 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:49.863230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:49.895350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:49.906797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1105/2213] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:49 2022] POST /identity/v3/users => generated 291 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:49.908637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:49.922600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489290b1-4d13-4c88-a4d7-50485bd77324 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:49.928281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:49.929111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.929467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.929826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.930707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:49.932638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:49.945515 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1109/2214] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:49.950577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:49.963026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2addbf-bb54-4658-95ad-4933fc40f594 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:49.968478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545a1d4-36be-48f7-a2d6-1f418b193281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:49.969283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a1d4-36be-48f7-a2d6-1f418b193281 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:49.969582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a1d4-36be-48f7-a2d6-1f418b193281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:49.969982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a1d4-36be-48f7-a2d6-1f418b193281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:49.970881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545a1d4-36be-48f7-a2d6-1f418b193281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:49.972780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545a1d4-36be-48f7-a2d6-1f418b193281 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:49.985851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1106/2215] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:49.989799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.002019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57bb3-76be-4e09-9748-94a6abdc1853 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.007360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.008352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.008683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.008967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] PATH_INFO: `/v3/projects/0857b01ccc10405dbec30d7d494d793e/users/3204237139c34f29ac4d245d182b147c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.031366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0857b01ccc10405dbec30d7d494d793e, user_id=3204237139c34f29ac4d245d182b147c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.036808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.059156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1110/2216] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:49 2022] PUT /identity/v3/projects/0857b01ccc10405dbec30d7d494d793e/users/3204237139c34f29ac4d245d182b147c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:50.070469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.070842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.071260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.104113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] MFA Rules not processed for user `3204237139c34f29ac4d245d182b147c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:50.105411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:50.193898 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1107/2217] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:50.199591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.273036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96013a44-5b63-43e8-9fda-b812793bfa68 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.278126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.278885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.279248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.279596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.280378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.282475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.315076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1111/2218] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:50 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:50.322968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.341293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac4098-47f9-48ef-a03c-e28899821851 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.421219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.421920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.422207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.422485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.423242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.425273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.450064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1108/2219] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:50 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:50.454654 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.470955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eac64-22a9-409c-8bc3-48997a3f3f47 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.476221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.476985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.477273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.477564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.478324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.480015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.492967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1112/2220] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:50.498090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.510189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de82d2-1c99-443a-8ddb-2bf1b923517d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.515908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.516934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.517263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.517652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] PATH_INFO: `/v3/projects/ca32ce8804ec4716874360d5e6b63331/users/8474f0f21dc4433792432f5acbf72245/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.540935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca32ce8804ec4716874360d5e6b63331, user_id=8474f0f21dc4433792432f5acbf72245, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.543824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.560008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1109/2221] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:50 2022] PUT /identity/v3/projects/ca32ce8804ec4716874360d5e6b63331/users/8474f0f21dc4433792432f5acbf72245/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:50.564291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.575722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c7c7-7fef-42ab-99fb-22ffa5c5d928 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.649908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aacb74-9d4e-4162-876a-3482d2508763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.650648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aacb74-9d4e-4162-876a-3482d2508763 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.651019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aacb74-9d4e-4162-876a-3482d2508763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.651298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aacb74-9d4e-4162-876a-3482d2508763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.652193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aacb74-9d4e-4162-876a-3482d2508763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.654478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aacb74-9d4e-4162-876a-3482d2508763 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.670374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1113/2222] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:50.675805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.690312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016fe55-abcd-45e1-a10e-8f5701c80bea admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.697522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.698565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.698950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.699321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] PATH_INFO: `/v3/projects/ca32ce8804ec4716874360d5e6b63331/users/8474f0f21dc4433792432f5acbf72245/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.701733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca32ce8804ec4716874360d5e6b63331, user_id=8474f0f21dc4433792432f5acbf72245, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.705488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.720307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1110/2223] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:50 2022] PUT /identity/v3/projects/ca32ce8804ec4716874360d5e6b63331/users/8474f0f21dc4433792432f5acbf72245/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:50.737362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.737783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.738189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.775546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] MFA Rules not processed for user `8474f0f21dc4433792432f5acbf72245`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:50.777809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:50.808209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:50.840977 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.844230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:50.890726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1114/2224] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:50.903782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:50.904891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:50.907103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:50.954978 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fea764-f07f-402c-ab1a-78cfdaa37e82 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:50.959818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:50.968178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:50.968178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:50.968178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] PATH_INFO: `/v3/users/69121a15a0714ff9a700f0be2580cbe1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:50.968178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69121a15a0714ff9a700f0be2580cbe1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:50.968178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:50.999684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ba3f-9c66-44d4-ba27-3271f111d5be None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:51.005663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2e34d-6950-479a-888a-ab8e50dcc489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:51.006607 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2e34d-6950-479a-888a-ab8e50dcc489 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.007038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2e34d-6950-479a-888a-ab8e50dcc489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.007434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2e34d-6950-479a-888a-ab8e50dcc489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.019908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2e34d-6950-479a-888a-ab8e50dcc489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:51.021474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2e34d-6950-479a-888a-ab8e50dcc489 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:51.032749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1115/2225] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:51.037170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69121a15a0714ff9a700f0be2580cbe1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.054118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69121a15a0714ff9a700f0be2580cbe1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.064211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69121a15a0714ff9a700f0be2580cbe1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.076539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69121a15a0714ff9a700f0be2580cbe1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.077595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69121a15a0714ff9a700f0be2580cbe1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.086901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1111/2226] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:50 2022] DELETE /identity/v3/users/69121a15a0714ff9a700f0be2580cbe1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:51.091603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.091909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.092195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.112425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:51.113670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.138772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:51.140324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.207020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1116/2227] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:51.221249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:51.221486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d61ea-12af-4e77-bdea-9a228f86c9f4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:51.224497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.226304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:51.227088 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.227338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.227886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.230122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.302601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:51.304105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:51.317039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1112/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:51.322780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362adf4-74c9-4318-9e39-560c91ff2276 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:51.328120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:51.328952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.329387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.329927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.345566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:51.347907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:51.368368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1117/2229] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:51.675552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:51.677138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.689391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:51.694453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bf159-519f-4bfb-8d87-7b42a367d497 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:51.704424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:51.705720 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.706272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.706785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] PATH_INFO: `/v3/users/3204237139c34f29ac4d245d182b147c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.708850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3204237139c34f29ac4d245d182b147c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:51.711160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:51.736896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:51.738487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.769457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:51.779352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741998f4-53e9-428b-accd-c034f985e81f service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:51.781635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3204237139c34f29ac4d245d182b147c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.788910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:51.790585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.791059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.791349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] PATH_INFO: `/v3/projects/17118e710bdc4116b5621d7333e69064` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.793366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17118e710bdc4116b5621d7333e69064)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:51.797377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:51.797553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3204237139c34f29ac4d245d182b147c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.814676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3204237139c34f29ac4d245d182b147c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.827113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3204237139c34f29ac4d245d182b147c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.827845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fff7341c-7ef2-4307-af7c-ce008f515847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3204237139c34f29ac4d245d182b147c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.843853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1113/2230] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:51 2022] DELETE /identity/v3/users/3204237139c34f29ac4d245d182b147c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:51.847837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:51.848198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:51.848579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:51.871460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:51.873248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.889851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17118e710bdc4116b5621d7333e69064', 'request_id': 'req-4dda8986-5457-44a9-9082-e66e0e6beb68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.896886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17118e710bdc4116b5621d7333e69064', 'request_id': 'req-4dda8986-5457-44a9-9082-e66e0e6beb68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.906291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17118e710bdc4116b5621d7333e69064', 'request_id': 'req-4dda8986-5457-44a9-9082-e66e0e6beb68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.918830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] The token cache is being invalidate because project 17118e710bdc4116b5621d7333e69064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:51.919120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.919787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:51.921543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1118/2231] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:51 2022] DELETE /identity/v3/projects/17118e710bdc4116b5621d7333e69064 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:51.928079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:51.929635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:51.960006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1114/2232] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:51.974888 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:51.976492 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:52.014676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda8986-5457-44a9-9082-e66e0e6beb68 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:52.019912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:52.020858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.021147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.021427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] PATH_INFO: `/v3/users/0d477c6e7987472292bfc43b32ce507d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.022724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d477c6e7987472292bfc43b32ce507d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:52.024748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:52.049027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff42d50-bc9c-4a5d-8201-fa491180ba84 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:52.053281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:52.053989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.054320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.054604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.065921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:52.067383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:52.077727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1115/2233] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:52.083539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d477c6e7987472292bfc43b32ce507d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.097922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d477c6e7987472292bfc43b32ce507d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.107990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d477c6e7987472292bfc43b32ce507d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.108889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d477c6e7987472292bfc43b32ce507d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.109593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d477c6e7987472292bfc43b32ce507d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.127330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1119/2234] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:51 2022] DELETE /identity/v3/users/0d477c6e7987472292bfc43b32ce507d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:52.408142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:52.420466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:52.460299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca0a01-0984-415b-bd28-c43da920d620 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:52.498815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:52.583816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:52.585971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.587099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.587639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] PATH_INFO: `/v3/projects/0857b01ccc10405dbec30d7d494d793e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.590254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0857b01ccc10405dbec30d7d494d793e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:52.594843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:52.610578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:52.612135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:52.629837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:52.635370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e0a46-f565-4311-951d-25b056ea7f29 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:52.650265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:52.651675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.652168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.652598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] PATH_INFO: `/v3/projects/d5513d44eab34d35a626ff9e4cebaafe` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.662956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5513d44eab34d35a626ff9e4cebaafe)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:52.665698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:52.685176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0857b01ccc10405dbec30d7d494d793e', 'request_id': 'req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.699013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0857b01ccc10405dbec30d7d494d793e', 'request_id': 'req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.710826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0857b01ccc10405dbec30d7d494d793e', 'request_id': 'req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.723404 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] The token cache is being invalidate because project 0857b01ccc10405dbec30d7d494d793e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:52.723682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.724448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.726082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1116/2235] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:52 2022] DELETE /identity/v3/projects/0857b01ccc10405dbec30d7d494d793e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:52.730765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5513d44eab34d35a626ff9e4cebaafe', 'request_id': 'req-a993094f-63fc-434f-a488-84fdb8d9217c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.731149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:52.733120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:52.746800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5513d44eab34d35a626ff9e4cebaafe', 'request_id': 'req-a993094f-63fc-434f-a488-84fdb8d9217c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.758610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5513d44eab34d35a626ff9e4cebaafe', 'request_id': 'req-a993094f-63fc-434f-a488-84fdb8d9217c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.771236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] The token cache is being invalidate because project d5513d44eab34d35a626ff9e4cebaafe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:52.771534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.772810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a993094f-63fc-434f-a488-84fdb8d9217c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:52.774499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1120/2236] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:52 2022] DELETE /identity/v3/projects/d5513d44eab34d35a626ff9e4cebaafe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:52.779894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db73371-6486-4635-a789-68fb7debf7bd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.780294 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db73371-6486-4635-a789-68fb7debf7bd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.780512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db73371-6486-4635-a789-68fb7debf7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.801930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1db73371-6486-4635-a789-68fb7debf7bd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:52.803234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1db73371-6486-4635-a789-68fb7debf7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:52.847328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25dbc8-d5ea-48a6-bb03-6839bfc3c568 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:52.892616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1121/2237] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:52.899679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.900091 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.900525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.925238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:52.927074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:52.927693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:52.935999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:52.937043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.937345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.938152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] PATH_INFO: `/v3/users/8474f0f21dc4433792432f5acbf72245` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.940208 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8474f0f21dc4433792432f5acbf72245)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:52.941896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1122/2238] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:52.942469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:52.948949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:52.961149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032896b-6fe8-41e6-8cd8-6548623f2de1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:52.966291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:52.966976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:52.967265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:52.967543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:52.968296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:52.968637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:52.970298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:52.988900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1123/2239] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:52.997449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.016007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4306d71-6c01-4cbd-8b46-0382ad369e4f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.025241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.026573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.027057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.027426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.028550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8474f0f21dc4433792432f5acbf72245'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.029395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.032857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.050937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8474f0f21dc4433792432f5acbf72245'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.064662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8474f0f21dc4433792432f5acbf72245'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.073068 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1124/2240] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:52 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:53.075506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.078850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8474f0f21dc4433792432f5acbf72245'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.079746 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8474f0f21dc4433792432f5acbf72245'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.090428 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1117/2241] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:52 2022] DELETE /identity/v3/users/8474f0f21dc4433792432f5acbf72245 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:53.164214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:53.179781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135c4df-061e-43d1-9357-d5cc2721bde1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.253816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.254649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.254940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.255233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.255933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.257943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.288623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1125/2242] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:53 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:53.293496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.295170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:53.316083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcddec6-de79-4cb2-9b23-f50b86593aba admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.327661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.328305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.329409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.329409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.329907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.331586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.343638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1118/2243] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:53.347382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.359134 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0bdd9-674c-49fb-8a06-0c00e94f040a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.364314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.365114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.365517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.365948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.366961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.369646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.381563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1126/2244] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:53.386187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.397982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeab67-0360-4d03-b560-35f115993bf4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.403604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60350d5c-6bea-4d58-a07a-e6c2cf510e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.404565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60350d5c-6bea-4d58-a07a-e6c2cf510e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.404853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60350d5c-6bea-4d58-a07a-e6c2cf510e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.405131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60350d5c-6bea-4d58-a07a-e6c2cf510e61 admin admin] PATH_INFO: `/v3/projects/675de382321f4798a8cafaa6c4ca66c0/users/0c9e08aac1f5491491e9752087a06c22/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.426829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60350d5c-6bea-4d58-a07a-e6c2cf510e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675de382321f4798a8cafaa6c4ca66c0, user_id=0c9e08aac1f5491491e9752087a06c22, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.432242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60350d5c-6bea-4d58-a07a-e6c2cf510e61 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.455271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1119/2245] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:53 2022] PUT /identity/v3/projects/675de382321f4798a8cafaa6c4ca66c0/users/0c9e08aac1f5491491e9752087a06c22/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:53.460020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.463970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:53.465037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.465334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.465690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.481234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9490200-3f53-45f6-b607-e16907a3f058 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.499807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] MFA Rules not processed for user `0c9e08aac1f5491491e9752087a06c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:53.501929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:53.571124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.572219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.572773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.573005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] PATH_INFO: `/v3/projects/ca32ce8804ec4716874360d5e6b63331` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.574646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca32ce8804ec4716874360d5e6b63331)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.577633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.640624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1120/2246] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:53.646718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.665340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf905b-5b40-4e34-b584-cae87ce9f8eb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.666559 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca32ce8804ec4716874360d5e6b63331', 'request_id': 'req-1321ff1d-9a58-43cb-96cb-dd501a078f73'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.679793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca32ce8804ec4716874360d5e6b63331', 'request_id': 'req-1321ff1d-9a58-43cb-96cb-dd501a078f73'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.693574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca32ce8804ec4716874360d5e6b63331', 'request_id': 'req-1321ff1d-9a58-43cb-96cb-dd501a078f73'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.714093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:53.718606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] The token cache is being invalidate because project ca32ce8804ec4716874360d5e6b63331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:53.718904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.720472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1321ff1d-9a58-43cb-96cb-dd501a078f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:53.721653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1127/2247] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:53 2022] DELETE /identity/v3/projects/ca32ce8804ec4716874360d5e6b63331 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:53.726749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873189c7-e501-4658-9a9a-db36bd872d2c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.727612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873189c7-e501-4658-9a9a-db36bd872d2c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.727897 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873189c7-e501-4658-9a9a-db36bd872d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.735043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:53.751822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-873189c7-e501-4658-9a9a-db36bd872d2c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:53.753105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-873189c7-e501-4658-9a9a-db36bd872d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:53.776884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.777879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.778170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.778454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.779161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.781183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.820612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1121/2248] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:53 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:53.821673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.823403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:53.843690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1128/2249] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:53.851466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.851752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.852843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.873811 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:53.875074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:53.913976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0697-697d-4b40-b388-07c92a3cf768 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.919540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.920343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.920869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.921285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.922018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.924022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.947507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1122/2250] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:53 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:53.953258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.956098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1129/2251] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:53.961061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:53.965280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f8587-9606-40d1-ae20-301f5c6edbec admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.970286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.971025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.971311 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.971593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.972363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.972848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6305b-b1ff-4438-9a9a-f44984eb2344 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:53.974044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.979165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:53.980436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:53.980753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:53.981247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:53.982463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:53.982933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:53.984954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:53.985585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1123/2252] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:53.990009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.000873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1130/2253] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:54.005270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af4922-0dc3-4cd7-9c90-8fde1daf1fb7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.006212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.006350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.007122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.007414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.007690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] PATH_INFO: `/v3/projects/38afa0b616074dabb01d892089b63d5a/users/0c345774fa744b7c89aaec2ac392a6c9/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.020149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a47fea-0dc0-4265-a439-1f9f8e80b9f8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.026813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38afa0b616074dabb01d892089b63d5a, user_id=0c345774fa744b7c89aaec2ac392a6c9, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.029840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.036373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.037365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.037868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.038138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.039439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.042191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.046336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1124/2254] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:53 2022] PUT /identity/v3/projects/38afa0b616074dabb01d892089b63d5a/users/0c345774fa744b7c89aaec2ac392a6c9/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:54.053216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.076428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1131/2255] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:54 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:54.081283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.161474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5fa5b-2917-4576-84c6-fda106c0ae94 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.167998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.168679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.168962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.169257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.170430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.172655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.190765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:54.200628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1132/2256] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:54 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:54.205263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.214683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1d0ca-55bb-4738-9163-bafe4ca76467 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.223037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690471a-7acf-4f64-bd6b-f0ef3c786e12 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.223376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.223848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.224909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.225111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.225438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.225636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.225999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.226164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.227832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.228038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.229744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.231555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.248817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1133/2257] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:54.249883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.255385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1125/2258] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:54.263337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.269443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36984884-5a9c-48e8-9f56-80d4ebdcd024 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.279645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.280421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ba226-42c4-49bf-aba9-f87da464bc78 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.281479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.282125 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.283018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.284564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.288577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbc400-32ce-4d72-9412-183a58574593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.288981 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.290822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbc400-32ce-4d72-9412-183a58574593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.291273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbc400-32ce-4d72-9412-183a58574593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.291754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbc400-32ce-4d72-9412-183a58574593 admin admin] PATH_INFO: `/v3/projects/38afa0b616074dabb01d892089b63d5a/users/0c345774fa744b7c89aaec2ac392a6c9/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.299925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbc400-32ce-4d72-9412-183a58574593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38afa0b616074dabb01d892089b63d5a, user_id=0c345774fa744b7c89aaec2ac392a6c9, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.306543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbc400-32ce-4d72-9412-183a58574593 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.310760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:54.315317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1134/2259] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:54.319392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.336828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1126/2260] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:54 2022] PUT /identity/v3/projects/38afa0b616074dabb01d892089b63d5a/users/0c345774fa744b7c89aaec2ac392a6c9/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:54.340708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223a19-88ac-4ddd-8ac2-be2b96a9f6ed admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.347668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.348143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.348638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.385546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] MFA Rules not processed for user `0c345774fa744b7c89aaec2ac392a6c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:54.387023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:54.447214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948590b-f0be-4bf4-a33c-1832d79c7c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.448292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948590b-f0be-4bf4-a33c-1832d79c7c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.449056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948590b-f0be-4bf4-a33c-1832d79c7c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.449646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948590b-f0be-4bf4-a33c-1832d79c7c92 admin admin] PATH_INFO: `/v3/projects/7ab4c638515744558f02a4c50813c0cd/users/3159dd4cd98f4c63b9f75c4b6c170990/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.481615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948590b-f0be-4bf4-a33c-1832d79c7c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab4c638515744558f02a4c50813c0cd, user_id=3159dd4cd98f4c63b9f75c4b6c170990, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.487384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948590b-f0be-4bf4-a33c-1832d79c7c92 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.516050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1135/2261] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:54 2022] PUT /identity/v3/projects/7ab4c638515744558f02a4c50813c0cd/users/3159dd4cd98f4c63b9f75c4b6c170990/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:54.522876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.524530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.524896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.542496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1127/2262] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:54.551416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:54.553409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:54.566424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.569089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:54.685113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae24d3-321d-4277-a88b-fa0c1aaf6195 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.691101 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1136/2263] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:54.704512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:54.706874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:54.803398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.804134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.804805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.805921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.847628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114b26e-4a32-490b-983e-290914e5a305 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:54.855827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea354e-60a1-429f-9922-01c8d753cb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:54.857296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea354e-60a1-429f-9922-01c8d753cb51 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:54.857813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea354e-60a1-429f-9922-01c8d753cb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:54.858323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea354e-60a1-429f-9922-01c8d753cb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:54.881202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea354e-60a1-429f-9922-01c8d753cb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.884953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea354e-60a1-429f-9922-01c8d753cb51 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.909111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:54.913183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1137/2264] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:54.946705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:54.949722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:54.978523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1128/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:55.942765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:55.943312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:55.943754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:55.951583 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:55.998196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] MFA Rules not processed for user `3159dd4cd98f4c63b9f75c4b6c170990`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:56.001108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:56.167356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.176081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1138/2266] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:56.181453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1404c-0959-4ff8-9dfc-3f2c72a457e4 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.187416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.188217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.188547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.188827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.189570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.191725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.199359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.212035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdefd3-e64a-47ce-9f1c-e47e624064ea None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.217317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.218239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.218522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.218800 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] PATH_INFO: `/v3/users/0c9e08aac1f5491491e9752087a06c22` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.220238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9e08aac1f5491491e9752087a06c22)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.222296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.239020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1129/2267] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:56 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:56.244803 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.314866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9e08aac1f5491491e9752087a06c22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:56.338146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9e08aac1f5491491e9752087a06c22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:56.351226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9e08aac1f5491491e9752087a06c22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:56.353109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9e08aac1f5491491e9752087a06c22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:56.354306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-81bd1665-4e86-4b34-95a7-604411891c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9e08aac1f5491491e9752087a06c22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:56.380178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42557843-e18f-4ea5-ae3f-0284b2184dd8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.390265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.391328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1139/2268] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:56 2022] DELETE /identity/v3/users/0c9e08aac1f5491491e9752087a06c22 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:56.392151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.392765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.393351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.394777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.398794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.399071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.399265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.399841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.429046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:56.432193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:56.457618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1130/2269] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:56 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:56.460123 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.464699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:56.488101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa051a62-c548-4208-aa84-a098e9bdfc99 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.558144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1140/2270] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:56.568530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.570197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:56.578697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:56.599411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.600562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.600906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.601192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.602135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.603953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.627524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1131/2271] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:56.629494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.649940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fade76f-13a5-4dff-81b5-a48de8cd3eb2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.652876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1e603-dfd9-4904-b4df-7f247b9d700d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.656850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.657525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.657810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.658031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.658145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.659580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.660094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.660639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] PATH_INFO: `/v3/projects/5c45c9e1a0394f1ea763b59d3bb7868e/users/6ef914af791843b0bcea99600070d885/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.670659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.672147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.680879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c45c9e1a0394f1ea763b59d3bb7868e, user_id=6ef914af791843b0bcea99600070d885, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.683789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.684144 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1141/2272] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:02:56.702133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1132/2273] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:56 2022] PUT /identity/v3/projects/5c45c9e1a0394f1ea763b59d3bb7868e/users/6ef914af791843b0bcea99600070d885/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:56.706316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.719553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8699fa4-eefd-4d5e-90de-e91848559eb3 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.820141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.821045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.821374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.821697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.822691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.825004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.847165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1142/2274] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:56.851884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:56.865987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f334ed9-eeff-4934-8835-1165ba12a471 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:56.874246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:56.875431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:56.875777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:56.876285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] PATH_INFO: `/v3/projects/5c45c9e1a0394f1ea763b59d3bb7868e/users/6ef914af791843b0bcea99600070d885/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:56.879214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c45c9e1a0394f1ea763b59d3bb7868e, user_id=6ef914af791843b0bcea99600070d885, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:56.883198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:56.897491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1133/2275] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:56 2022] PUT /identity/v3/projects/5c45c9e1a0394f1ea763b59d3bb7868e/users/6ef914af791843b0bcea99600070d885/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:57.161646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:57.163621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:57.179414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee86db-2500-441b-af34-0e5550bcf72b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:57.274905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:57.276007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:57.276472 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:57.276926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] PATH_INFO: `/v3/projects/675de382321f4798a8cafaa6c4ca66c0` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:57.278813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] RBAC: Authorizing `identity:delete_project(project_id=675de382321f4798a8cafaa6c4ca66c0)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:57.281350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:57.361062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:57.367889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '675de382321f4798a8cafaa6c4ca66c0', 'request_id': 'req-071da3bc-8b14-4104-af84-2cc26822013a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.382592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '675de382321f4798a8cafaa6c4ca66c0', 'request_id': 'req-071da3bc-8b14-4104-af84-2cc26822013a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.396612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '675de382321f4798a8cafaa6c4ca66c0', 'request_id': 'req-071da3bc-8b14-4104-af84-2cc26822013a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.414523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] The token cache is being invalidate because project 675de382321f4798a8cafaa6c4ca66c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:57.415161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.416842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-071da3bc-8b14-4104-af84-2cc26822013a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.421734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1143/2276] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:57 2022] DELETE /identity/v3/projects/675de382321f4798a8cafaa6c4ca66c0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:57.423335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:57.426821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:57.474831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1ca8a-983d-45ff-80d1-1d7edbbc177c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:57.487117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:57.616589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:57.617530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:57.617817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:57.618253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] PATH_INFO: `/v3/users/0c345774fa744b7c89aaec2ac392a6c9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:57.619572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c345774fa744b7c89aaec2ac392a6c9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:57.621801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:57.692096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c345774fa744b7c89aaec2ac392a6c9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.719642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c345774fa744b7c89aaec2ac392a6c9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.733434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c345774fa744b7c89aaec2ac392a6c9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.750411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c345774fa744b7c89aaec2ac392a6c9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.751907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c345774fa744b7c89aaec2ac392a6c9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:57.765589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1134/2277] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:02:57 2022] DELETE /identity/v3/users/0c345774fa744b7c89aaec2ac392a6c9 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:58.346586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:58.347391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:58.347577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:58.397954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] MFA Rules not processed for user `6ef914af791843b0bcea99600070d885`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:58.399193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:58.532634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1144/2278] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:58 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:58.542322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:58.544836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:58.590517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:58.594264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:58.691529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc2b61-c2a8-42fe-be57-8583c4658e6d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:58.696621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:58.697401 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:58.697788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:58.698156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:58.700805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:58.731196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660602b-46ed-4e8b-9fc4-1cf4df7543d3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:58.737946 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:58.738992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:58.739380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:58.739762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] PATH_INFO: `/v3/projects/38afa0b616074dabb01d892089b63d5a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:58.741157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38afa0b616074dabb01d892089b63d5a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:58.743870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:58.753998 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:58.771769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:58.773263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb919d9-c867-4ff6-a652-101780e6c980 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:58.784035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1135/2279] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:02:58.816029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38afa0b616074dabb01d892089b63d5a', 'request_id': 'req-ead2de22-4908-4a3e-9240-6d1cc9329699'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:58.830729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38afa0b616074dabb01d892089b63d5a', 'request_id': 'req-ead2de22-4908-4a3e-9240-6d1cc9329699'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:58.845446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38afa0b616074dabb01d892089b63d5a', 'request_id': 'req-ead2de22-4908-4a3e-9240-6d1cc9329699'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:58.847296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:58.862322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] The token cache is being invalidate because project 38afa0b616074dabb01d892089b63d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:02:58.863178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:58.864469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ead2de22-4908-4a3e-9240-6d1cc9329699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:02:58.866615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1145/2280] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:02:58 2022] DELETE /identity/v3/projects/38afa0b616074dabb01d892089b63d5a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:02:58.871875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:58.872263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:58.872652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:58.900966 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:58.902979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:59.009738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1136/2281] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:59.084662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.084976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.085350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.115815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:59.118641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:59.140213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1146/2282] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:02:59.145275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.168369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45940df9-be13-4db5-bd29-95dbf12e838d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.177659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.179650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.180795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.181297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.182821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:02:59.183270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.186688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.215618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1137/2283] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:02:59.217500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.228492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88a265-37d5-495a-9d33-3bec973a6aa8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.235529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.236227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.236688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.236884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.237765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.240100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.281442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1147/2284] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:02:59 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:02:59.283427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.303170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59105ae2-f2d9-40af-ad73-d8d0201dd1d5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.398009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.400050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.400621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.401146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.402765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.406650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.437666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1138/2285] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:02:59 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:02:59.443072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.454624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b860eb-a0b8-43d0-a672-dcce0936aa8e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.459954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.460687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.460976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.461256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.462012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.463671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.477753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1148/2286] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:02:59.483579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.495713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5637d67-2fac-4d36-abef-f47ec69f1e7e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.502825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e93a85-7999-4797-a395-c9661c4c5eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.504370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93a85-7999-4797-a395-c9661c4c5eae admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.505902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93a85-7999-4797-a395-c9661c4c5eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.506509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e93a85-7999-4797-a395-c9661c4c5eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.508482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e93a85-7999-4797-a395-c9661c4c5eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.511582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e93a85-7999-4797-a395-c9661c4c5eae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.536033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1139/2287] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:02:59.541272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.553400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639de2a-247f-45e4-bde0-0dcfc62b43dc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.560009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.561075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.561569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.561973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] PATH_INFO: `/v3/projects/abe72bdce5f84a06adf83804a7f68919/users/f6084615eaf94ddc9bb8b594923b8bf9/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.588895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe72bdce5f84a06adf83804a7f68919, user_id=f6084615eaf94ddc9bb8b594923b8bf9, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.594364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.613067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1149/2288] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:02:59 2022] PUT /identity/v3/projects/abe72bdce5f84a06adf83804a7f68919/users/f6084615eaf94ddc9bb8b594923b8bf9/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:02:59.624976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.625537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.625704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.660672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] MFA Rules not processed for user `f6084615eaf94ddc9bb8b594923b8bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:02:59.661955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:59.724451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:02:59.759769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1140/2289] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:02:59.774568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:02:59.776697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:02:59.883768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6459-3e42-4e47-be6a-8de3ddc592c8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:02:59.890194 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:02:59.891228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:02:59.891846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:02:59.892317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:02:59.913074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:02:59.915129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:02:59.930638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1150/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:02:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:03:00.322017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:00.326083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:00.405503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa3b2c-9de6-4ec1-a6c2-46626f102264 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:00.409251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:00.410115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:00.410416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:00.410732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:00.413226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:00.505351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:00.507398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d486e0d-4edc-424b-b028-a2f8cce4358f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:00.548954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1141/2291] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:00 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:01.049391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:01.051190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:01.174835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef190254-38cd-4730-b50d-8d90904359ec service glance] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:01.183838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:01.185552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:01.186140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:01.186722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] PATH_INFO: `/v3/users/44ab217336e64971a91f4cd19f92eac7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:01.189615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ab217336e64971a91f4cd19f92eac7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:01.192931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:01.270357 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:01.288039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ab217336e64971a91f4cd19f92eac7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:01.307694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ab217336e64971a91f4cd19f92eac7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:01.319424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ab217336e64971a91f4cd19f92eac7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:01.320220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ab217336e64971a91f4cd19f92eac7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:01.321601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ab217336e64971a91f4cd19f92eac7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:01.339809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1151/2292] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:01 2022] DELETE /identity/v3/users/44ab217336e64971a91f4cd19f92eac7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:01.346354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:01.346354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:01.346354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:01.366828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:01.368242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:01.465192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1142/2293] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:01.475854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:01.478589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:01.577259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa3ec5-ae55-45fd-9f6e-b9e87b2f9dcb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:01.583103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc9511-d862-40e1-afd4-235e3c56de0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:01.583904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc9511-d862-40e1-afd4-235e3c56de0d service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:01.584291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc9511-d862-40e1-afd4-235e3c56de0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:01.584771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc9511-d862-40e1-afd4-235e3c56de0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:01.600556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc9511-d862-40e1-afd4-235e3c56de0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:01.602530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc9511-d862-40e1-afd4-235e3c56de0d service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:01.620723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1152/2294] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:03:02.148056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:02.151172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:02.180779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc82fb0-e44e-4981-9a74-74c989ea06d6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:02.196334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:02.197794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.198266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.199302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] PATH_INFO: `/v3/projects/c29dae21c3444b21afab84a9df540725` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.201381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29dae21c3444b21afab84a9df540725)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:02.205239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:02.258551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:02.296181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29dae21c3444b21afab84a9df540725', 'request_id': 'req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:02.310487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29dae21c3444b21afab84a9df540725', 'request_id': 'req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:02.323995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29dae21c3444b21afab84a9df540725', 'request_id': 'req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:02.352220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] The token cache is being invalidate because project c29dae21c3444b21afab84a9df540725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:02.353145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:02.354871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95629c6c-d8c4-4b13-bd76-a46f13b8f1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:02.360962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1143/2295] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:02 2022] DELETE /identity/v3/projects/c29dae21c3444b21afab84a9df540725 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:02.364821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.365761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.366606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.408604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:02.411561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:02.543247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1153/2296] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:02.558398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.558703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.559025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.583114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:02.585315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:02.599221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1144/2297] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:02.603405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:02.615865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac455820-b2f4-4ea2-a1c3-d297794458d9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:02.622415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:02.623143 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.623432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.623706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.624732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:03:02.624882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:02.626498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:02.642510 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1154/2298] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:03:02.646893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:02.668694 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac8b5d-aecf-4ae5-9996-267e0995d87d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:02.685758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:02.687038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.687781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.688529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.690497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:02.693348 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:02.731644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1145/2299] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:03:02 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:02.738288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:02.752529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60a38e-d0b4-4d75-b00b-a7ea5aa5093e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:02.808020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:02.859900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:02.861287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.861931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.862593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.864141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:02.868903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:02.897124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1155/2300] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:03:02 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:02.903043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:02.927754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0487dda-3664-4971-9c3b-b79fadf9b000 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:02.936503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:02.937713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.938207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.938820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.940224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:02.943405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:02.957111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1146/2301] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:02.962873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:02.979453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6203fa3-9285-4999-b74a-404fcda7f49a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:02.987615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa984222-007a-4062-8c98-59d0c30c0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:02.988786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984222-007a-4062-8c98-59d0c30c0252 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:02.989290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984222-007a-4062-8c98-59d0c30c0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:02.989789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984222-007a-4062-8c98-59d0c30c0252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:02.991278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa984222-007a-4062-8c98-59d0c30c0252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:02.995307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa984222-007a-4062-8c98-59d0c30c0252 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:03.017887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1156/2302] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:03:03.022308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:03.039921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb1d5a-59e0-4506-ae22-e17c1d971508 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:03.048115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:03.049772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:03.050288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:03.050790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] PATH_INFO: `/v3/projects/8969431739ca45c4a0adcaf6f4891c3d/users/06fa8faf31264fdda109b3af3e9e080f/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:03.080724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8969431739ca45c4a0adcaf6f4891c3d, user_id=06fa8faf31264fdda109b3af3e9e080f, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:03.083645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:03.104455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1147/2303] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:03 2022] PUT /identity/v3/projects/8969431739ca45c4a0adcaf6f4891c3d/users/06fa8faf31264fdda109b3af3e9e080f/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:03.124984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:03.125327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:03.125614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:03.167942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] MFA Rules not processed for user `06fa8faf31264fdda109b3af3e9e080f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:03.170155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:03.304523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1157/2304] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:03.309287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:03.325518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:03.433868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60030f76-b0fe-49cd-9211-35964c3508f3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:03.441289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fe9c4-f137-4514-8f4b-868e08fe0398 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:03.442437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe9c4-f137-4514-8f4b-868e08fe0398 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:03.442936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe9c4-f137-4514-8f4b-868e08fe0398 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:03.443427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fe9c4-f137-4514-8f4b-868e08fe0398 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:03.460039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fe9c4-f137-4514-8f4b-868e08fe0398 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:03.462022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fe9c4-f137-4514-8f4b-868e08fe0398 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:03.478912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1148/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:11.833280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:11.834879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:11.918651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f240-a0e8-479b-b2e0-774923c9c315 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:11.923362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:11.924266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:11.924586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:11.924869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] PATH_INFO: `/v3/users/cef145adfa384ef7878ce4d612996e70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:11.926196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef145adfa384ef7878ce4d612996e70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:11.928162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:11.989865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef145adfa384ef7878ce4d612996e70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:11.991288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:12.009199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef145adfa384ef7878ce4d612996e70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:12.019261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef145adfa384ef7878ce4d612996e70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:12.019934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef145adfa384ef7878ce4d612996e70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:12.020878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef145adfa384ef7878ce4d612996e70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:12.041502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1158/2306] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:11 2022] DELETE /identity/v3/users/cef145adfa384ef7878ce4d612996e70 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:12.047832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:12.048499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:12.048753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:12.071982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:12.074586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:12.129700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:12.147969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1149/2307] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:12.159427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:12.161045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:12.231036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0ab52-c1fe-46d4-ba5e-cfd3a44499d4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:12.240634 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b0869-9519-40a9-a06c-35cfdacccffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:12.241870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b0869-9519-40a9-a06c-35cfdacccffc service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:12.242487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b0869-9519-40a9-a06c-35cfdacccffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:12.243093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b0869-9519-40a9-a06c-35cfdacccffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:12.263530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b0869-9519-40a9-a06c-35cfdacccffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:12.265952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b0869-9519-40a9-a06c-35cfdacccffc service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:12.281051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1159/2308] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:12.779296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:12.782061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:12.832989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437c379-b1b4-4c37-b49a-dcd54dd69702 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:12.858971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:12.863641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:12.864118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:12.868472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] PATH_INFO: `/v3/projects/977274af64b2438db7b87ef9dbf18ea5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:12.877687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977274af64b2438db7b87ef9dbf18ea5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:12.889292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.081869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977274af64b2438db7b87ef9dbf18ea5', 'request_id': 'req-9abd1961-617a-438e-ba00-2022697a6fe0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:13.101202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977274af64b2438db7b87ef9dbf18ea5', 'request_id': 'req-9abd1961-617a-438e-ba00-2022697a6fe0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:13.122147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977274af64b2438db7b87ef9dbf18ea5', 'request_id': 'req-9abd1961-617a-438e-ba00-2022697a6fe0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:13.140564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] The token cache is being invalidate because project 977274af64b2438db7b87ef9dbf18ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:13.140864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:13.142446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9abd1961-617a-438e-ba00-2022697a6fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:13.149246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1150/2309] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:12 2022] DELETE /identity/v3/projects/977274af64b2438db7b87ef9dbf18ea5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:03:13.156128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.158242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.169062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.191982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:13.193282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:13.310664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1160/2310] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:13.319030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.319423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.319702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.350276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:13.352618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:13.369932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1151/2311] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:13.376014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:13.394285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d667490-9bbe-4266-90f4-848c8a8deb4f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:13.403137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:13.404799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.405309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.406006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.407706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:03:13.408295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:13.412784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.429199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1161/2312] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:03:13.436153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:13.454336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8719-e0fc-4c01-a481-30d159bb6be8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:13.462441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:13.464243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.464792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.465286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.466617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:13.470487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.514738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1152/2313] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:03:13 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:13.519590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:13.536151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1f354-9831-4dee-af6a-25be3948d918 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:13.600605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:13.613406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:13.614486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.615228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.615972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.617345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:13.620789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.655749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1162/2314] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:03:13 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:03:13.660214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:13.672182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26e61-231f-4c5f-9f8e-4f05393dfb4a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:13.677057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:13.677895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.678178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.678468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.679219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:13.680904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.693103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1153/2315] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:13.698955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:13.717879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9058-26ec-413f-bd3d-ef53b6717841 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:13.725969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49453df0-d8dd-4c0c-bf85-68090dd75dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:13.727118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49453df0-d8dd-4c0c-bf85-68090dd75dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.727601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49453df0-d8dd-4c0c-bf85-68090dd75dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.728100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49453df0-d8dd-4c0c-bf85-68090dd75dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.729525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49453df0-d8dd-4c0c-bf85-68090dd75dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:13.732860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49453df0-d8dd-4c0c-bf85-68090dd75dc6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.753287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1163/2316] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:13.757802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:13.772748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cde5dd-cef5-4a94-a7f3-064ac25f2b29 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:13.781025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:13.782517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.782992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.783462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] PATH_INFO: `/v3/projects/2c19761fa61741768b82406c6d912f36/users/bf5bc5d19bf5478aa4422a8a03dd3abf/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.816355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c19761fa61741768b82406c6d912f36, user_id=bf5bc5d19bf5478aa4422a8a03dd3abf, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:13.821624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:13.839769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1154/2317] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:13 2022] PUT /identity/v3/projects/2c19761fa61741768b82406c6d912f36/users/bf5bc5d19bf5478aa4422a8a03dd3abf/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:13.859015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:13.859261 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:13.859536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:13.894880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] MFA Rules not processed for user `bf5bc5d19bf5478aa4422a8a03dd3abf`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:13.896128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:13.983391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1164/2318] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:13.998422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:14.001380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:14.078447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e332-b720-4a8e-b985-312289b58b46 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:14.084605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe866d11-6ec2-48ba-bc23-27c56051aca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:14.085750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe866d11-6ec2-48ba-bc23-27c56051aca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:14.086252 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe866d11-6ec2-48ba-bc23-27c56051aca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:14.086594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe866d11-6ec2-48ba-bc23-27c56051aca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:14.104056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe866d11-6ec2-48ba-bc23-27c56051aca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:14.106473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe866d11-6ec2-48ba-bc23-27c56051aca0 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:14.121724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1155/2319] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:14.156681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:14.279610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234e24a-b16d-4189-b015-c619e9d4b8c4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:14.293025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:14.294358 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:14.294650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:14.295036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] PATH_INFO: `/v3/users/bf5bc5d19bf5478aa4422a8a03dd3abf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:14.296579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5bc5d19bf5478aa4422a8a03dd3abf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:14.298627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:14.399301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5bc5d19bf5478aa4422a8a03dd3abf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:14.413734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5bc5d19bf5478aa4422a8a03dd3abf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:14.415796 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:14.426716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5bc5d19bf5478aa4422a8a03dd3abf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:14.427989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5bc5d19bf5478aa4422a8a03dd3abf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:14.429835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5bc5d19bf5478aa4422a8a03dd3abf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:14.449208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1165/2320] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:14 2022] DELETE /identity/v3/users/bf5bc5d19bf5478aa4422a8a03dd3abf => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:14.454921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:14.455386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:14.455783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:14.480274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:14.481641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:14.492704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:14.563011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1156/2321] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:14.574382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:14.575903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:14.695165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6de1e-4279-4b13-ae49-17c6a066db6b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:14.702669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:14.703823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:14.704330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:14.704840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:14.721687 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:14.724268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:14.742167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1166/2322] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:15.127547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:15.131186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:15.150066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d313e19-4b66-4d77-8aa8-6eff90f5a47c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:15.157360 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:15.158399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:15.158881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:15.159871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] PATH_INFO: `/v3/projects/2c19761fa61741768b82406c6d912f36` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:15.162436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c19761fa61741768b82406c6d912f36)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:15.166677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:15.233580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c19761fa61741768b82406c6d912f36', 'request_id': 'req-a8cb7126-2bf2-4150-bea3-29d032d0d676'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:15.242074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c19761fa61741768b82406c6d912f36', 'request_id': 'req-a8cb7126-2bf2-4150-bea3-29d032d0d676'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:15.252067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c19761fa61741768b82406c6d912f36', 'request_id': 'req-a8cb7126-2bf2-4150-bea3-29d032d0d676'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:15.266513 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] The token cache is being invalidate because project 2c19761fa61741768b82406c6d912f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:15.266815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:15.267701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:15.269585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1157/2323] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:15 2022] DELETE /identity/v3/projects/2c19761fa61741768b82406c6d912f36 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:17.458938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:17.461163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:17.481128 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a4fb-3feb-4564-abfa-bbbd238f059c service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:17.568865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:17.569786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:17.570070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:17.570351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] PATH_INFO: `/v3/users/3159dd4cd98f4c63b9f75c4b6c170990` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:17.571767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3159dd4cd98f4c63b9f75c4b6c170990)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:17.574401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:17.642629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3159dd4cd98f4c63b9f75c4b6c170990'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:17.657579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3159dd4cd98f4c63b9f75c4b6c170990'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:17.668597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3159dd4cd98f4c63b9f75c4b6c170990'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:17.669541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3159dd4cd98f4c63b9f75c4b6c170990'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:17.670273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3159dd4cd98f4c63b9f75c4b6c170990'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:17.686509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1167/2324] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:17 2022] DELETE /identity/v3/users/3159dd4cd98f4c63b9f75c4b6c170990 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:17.898902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:17.901648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:17.926285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb7126-2bf2-4150-bea3-29d032d0d676 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:18.024215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:18.025386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:18.025712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:18.027007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] PATH_INFO: `/v3/projects/7ab4c638515744558f02a4c50813c0cd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:18.028502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab4c638515744558f02a4c50813c0cd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:18.031878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:18.110219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:18.118485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab4c638515744558f02a4c50813c0cd', 'request_id': 'req-bd07fa59-9ef1-40a4-a3f9-7084200aad54'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.127250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab4c638515744558f02a4c50813c0cd', 'request_id': 'req-bd07fa59-9ef1-40a4-a3f9-7084200aad54'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.135690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab4c638515744558f02a4c50813c0cd', 'request_id': 'req-bd07fa59-9ef1-40a4-a3f9-7084200aad54'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.147515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] The token cache is being invalidate because project 7ab4c638515744558f02a4c50813c0cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:18.147811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.148869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.151555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1158/2325] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:17 2022] DELETE /identity/v3/projects/7ab4c638515744558f02a4c50813c0cd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:18.155790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:18.157277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:18.167661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:18.175938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77ba34-1e3f-4ec5-b985-2a2864d3c9eb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:18.262157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:18.263054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:18.263338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:18.263611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] PATH_INFO: `/v3/users/6ef914af791843b0bcea99600070d885` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:18.264859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef914af791843b0bcea99600070d885)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:18.266817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:18.319178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef914af791843b0bcea99600070d885'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.331267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef914af791843b0bcea99600070d885'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.339393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef914af791843b0bcea99600070d885'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.340030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef914af791843b0bcea99600070d885'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.340653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09fa989f-cc98-4eb9-ad40-eed429d86a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef914af791843b0bcea99600070d885'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.353219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1168/2326] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:18 2022] DELETE /identity/v3/users/6ef914af791843b0bcea99600070d885 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:18.570454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:18.574700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:18.608923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07fa59-9ef1-40a4-a3f9-7084200aad54 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:18.690450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:18.692216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:18.692772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:18.693303 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] PATH_INFO: `/v3/projects/5c45c9e1a0394f1ea763b59d3bb7868e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:18.695601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c45c9e1a0394f1ea763b59d3bb7868e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:18.701711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:18.772365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c45c9e1a0394f1ea763b59d3bb7868e', 'request_id': 'req-27d0bcda-8278-41c1-8d14-dae129c054ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.779755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c45c9e1a0394f1ea763b59d3bb7868e', 'request_id': 'req-27d0bcda-8278-41c1-8d14-dae129c054ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.790117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c45c9e1a0394f1ea763b59d3bb7868e', 'request_id': 'req-27d0bcda-8278-41c1-8d14-dae129c054ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.801472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] The token cache is being invalidate because project 5c45c9e1a0394f1ea763b59d3bb7868e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:18.801768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.802444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0bcda-8278-41c1-8d14-dae129c054ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:18.804025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1159/2327] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:18 2022] DELETE /identity/v3/projects/5c45c9e1a0394f1ea763b59d3bb7868e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:18.810071 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bec574-decc-41e7-a10a-9af502956975 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:18.810561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bec574-decc-41e7-a10a-9af502956975 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:18.810673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bec574-decc-41e7-a10a-9af502956975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:18.829417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-73bec574-decc-41e7-a10a-9af502956975 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:18.830650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-73bec574-decc-41e7-a10a-9af502956975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:18.902826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1169/2328] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:18.911246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:18.912064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:18.912209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:18.943652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:18.944842 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:18.953507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1160/2329] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:18.957951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bec574-decc-41e7-a10a-9af502956975 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:18.968565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bec574-decc-41e7-a10a-9af502956975 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:18.973315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:18.974018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:18.974278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:18.974591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:18.975336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:03:18.975675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:18.977462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:18.992307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1170/2330] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:03:18.997446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:19.007315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6baee2-5ce3-461e-92e3-f9401a29589c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:19.014221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:19.014919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.015204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.015484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.016427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:19.018578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:19.049242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1161/2331] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:03:18 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:19.054572 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:19.071393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61ee5-c2e9-4c3b-a27e-932132fbcde5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:19.082944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:19.154504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:19.155768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.156286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.156845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.158238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:19.168533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:19.200626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1171/2332] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:03:19 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:03:19.204627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:19.230113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1cdfb-3bb9-4a21-b28a-46139e3834ee admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:19.240731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:19.241581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.241853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.242301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.243798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:19.246105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:19.276743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1162/2333] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:19.281699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:19.307341 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003bed9-a41a-4e71-92be-cb32ab1c1a42 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:19.323424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9c88b-d2cd-42be-9cd7-87e3fbc5573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:19.324963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9c88b-d2cd-42be-9cd7-87e3fbc5573d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.325463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9c88b-d2cd-42be-9cd7-87e3fbc5573d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.325958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9c88b-d2cd-42be-9cd7-87e3fbc5573d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.327252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9c88b-d2cd-42be-9cd7-87e3fbc5573d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:19.330416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9c88b-d2cd-42be-9cd7-87e3fbc5573d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:19.350049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1172/2334] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:19.354328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:19.367346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee207f7-2767-4ca8-ba89-c28d12b30372 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:19.377083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:19.378125 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.378411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.378695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] PATH_INFO: `/v3/projects/f6d014f3503342658e8a04969b54122f/users/77ab1a7b561f47bebb26b89e3fdc561d/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.401923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d014f3503342658e8a04969b54122f, user_id=77ab1a7b561f47bebb26b89e3fdc561d, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:19.404907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:19.410040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:19.430629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1163/2335] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:19 2022] PUT /identity/v3/projects/f6d014f3503342658e8a04969b54122f/users/77ab1a7b561f47bebb26b89e3fdc561d/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:03:19.443029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.443327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.443612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.473099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] MFA Rules not processed for user `77ab1a7b561f47bebb26b89e3fdc561d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:19.474400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:19.552413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1173/2336] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:19.567649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:19.569530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:19.654392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced671c-56fe-4775-a61c-39435fbf9025 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:19.658168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859e1cd-c779-4f93-8dd0-0be1c97110e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:19.658831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859e1cd-c779-4f93-8dd0-0be1c97110e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:19.659192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859e1cd-c779-4f93-8dd0-0be1c97110e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:19.659567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859e1cd-c779-4f93-8dd0-0be1c97110e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:19.670356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859e1cd-c779-4f93-8dd0-0be1c97110e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:19.671869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859e1cd-c779-4f93-8dd0-0be1c97110e8 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:19.687010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1164/2337] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:03:23.125485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:23.208650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a013-54af-4a62-9505-f6f9dca19d24 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:23.213274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:23.214245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:23.214455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:23.214736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] PATH_INFO: `/v3/users/77ab1a7b561f47bebb26b89e3fdc561d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:23.216040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ab1a7b561f47bebb26b89e3fdc561d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:23.218105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:23.247178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:23.281489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ab1a7b561f47bebb26b89e3fdc561d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:23.296593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ab1a7b561f47bebb26b89e3fdc561d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:23.306160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ab1a7b561f47bebb26b89e3fdc561d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:23.306883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ab1a7b561f47bebb26b89e3fdc561d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:23.308237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ab1a7b561f47bebb26b89e3fdc561d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:23.321473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1174/2338] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:23 2022] DELETE /identity/v3/users/77ab1a7b561f47bebb26b89e3fdc561d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:23.326004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:23.326479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:23.326574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:23.350335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:23.351573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:23.418994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1165/2339] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:23.429474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:23.431614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:23.506872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058eb59-8aba-484d-839e-5f0aedb1a603 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:23.510763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:23.511435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:23.511722 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:23.511999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:23.522465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:23.523939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:23.535305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1175/2340] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:24.007328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:24.016209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:24.035319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec0a71-9226-4dfc-91e3-85d8d1753f80 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:24.065085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:24.066480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:24.066953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:24.067413 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] PATH_INFO: `/v3/projects/f6d014f3503342658e8a04969b54122f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:24.069837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d014f3503342658e8a04969b54122f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:24.074277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:24.155334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d014f3503342658e8a04969b54122f', 'request_id': 'req-ef9f9132-eb37-4c53-ab52-13e47e944ead'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:24.166367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d014f3503342658e8a04969b54122f', 'request_id': 'req-ef9f9132-eb37-4c53-ab52-13e47e944ead'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:24.180091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d014f3503342658e8a04969b54122f', 'request_id': 'req-ef9f9132-eb37-4c53-ab52-13e47e944ead'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:24.209263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] The token cache is being invalidate because project f6d014f3503342658e8a04969b54122f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:24.209674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:24.210828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ef9f9132-eb37-4c53-ab52-13e47e944ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:24.214478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1166/2341] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:24 2022] DELETE /identity/v3/projects/f6d014f3503342658e8a04969b54122f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:27.690507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:27.692031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:27.708666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8b7dd-2080-4e0b-a9ca-c1d375a0c586 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:27.775153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:27.776677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:27.776955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:27.777228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] PATH_INFO: `/v3/users/f6084615eaf94ddc9bb8b594923b8bf9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:27.778746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6084615eaf94ddc9bb8b594923b8bf9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:27.780736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:27.836733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:27.842029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6084615eaf94ddc9bb8b594923b8bf9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:27.856178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6084615eaf94ddc9bb8b594923b8bf9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:27.864220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6084615eaf94ddc9bb8b594923b8bf9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:27.864870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6084615eaf94ddc9bb8b594923b8bf9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:27.865556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6084615eaf94ddc9bb8b594923b8bf9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:27.879956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1176/2342] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:27 2022] DELETE /identity/v3/users/f6084615eaf94ddc9bb8b594923b8bf9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:27.885912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:27.886419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:27.886419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:27.905340 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:27.906561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:27.923633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:27.972270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1167/2343] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:27.982861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:27.984414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:28.046968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9da63-7668-43f5-855e-2f4cbf7456fe admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:28.050651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb2279-f39e-4db1-a2bc-218bb39de11c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:28.051304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb2279-f39e-4db1-a2bc-218bb39de11c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.051594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb2279-f39e-4db1-a2bc-218bb39de11c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.051876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb2279-f39e-4db1-a2bc-218bb39de11c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.061867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb2279-f39e-4db1-a2bc-218bb39de11c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:28.063330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb2279-f39e-4db1-a2bc-218bb39de11c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:28.074562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1177/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:28.407200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:28.409582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:28.429060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d3c59-e737-4ddd-8a7b-79c78a190f8f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:28.439663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:28.440603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.441047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.441525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] PATH_INFO: `/v3/projects/abe72bdce5f84a06adf83804a7f68919` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.443058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe72bdce5f84a06adf83804a7f68919)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:28.445167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:28.513115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe72bdce5f84a06adf83804a7f68919', 'request_id': 'req-1cdbbac5-f500-4d20-b498-5bab7a7483ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:28.521682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe72bdce5f84a06adf83804a7f68919', 'request_id': 'req-1cdbbac5-f500-4d20-b498-5bab7a7483ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:28.529934 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe72bdce5f84a06adf83804a7f68919', 'request_id': 'req-1cdbbac5-f500-4d20-b498-5bab7a7483ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:28.541915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] The token cache is being invalidate because project abe72bdce5f84a06adf83804a7f68919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:28.542223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:28.542869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1cdbbac5-f500-4d20-b498-5bab7a7483ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:28.544636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1168/2345] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:28 2022] DELETE /identity/v3/projects/abe72bdce5f84a06adf83804a7f68919 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:28.551582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.551862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.552137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.572067 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:28.573400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:28.641998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1178/2346] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:28.650522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.650773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.651000 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.670654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:28.672170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:28.681157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1169/2347] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:28.685991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:28.696272 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175f6ef-8e46-408e-b1ab-101b6a29cdd9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:28.708414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:28.709109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.709393 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.709668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.710407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:03:28.710711 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:28.712348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:28.729879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1179/2348] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:03:28.731083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:28.755330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fd21-7b02-4c9d-9dce-1fa957289b64 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:28.762034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:28.762891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.763283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.763589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.764556 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:28.766771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:28.809433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1170/2349] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:03:28 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:28.814234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:28.826868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9618c9-9247-4ebb-b1b4-98f98276d82f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:28.910281 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:28.929953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:28.930646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.930932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.931231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.932509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:28.935720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:28.969538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1180/2350] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:03:28 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:28.975747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:28.988646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abf04f-add4-4558-a79a-4e3ffb86d44b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:28.996425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:28.997277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:28.997568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:28.997877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:28.998630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:29.000279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:29.013670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1171/2351] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:29.018585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:29.035301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cebc-7785-478e-b430-83857cf89da5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:29.044880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e8cd-e793-45b7-81d2-678efd12ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:29.045640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e8cd-e793-45b7-81d2-678efd12ba6b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:29.047364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e8cd-e793-45b7-81d2-678efd12ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:29.047666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e8cd-e793-45b7-81d2-678efd12ba6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:29.048541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29e8cd-e793-45b7-81d2-678efd12ba6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:29.050442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29e8cd-e793-45b7-81d2-678efd12ba6b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:29.069021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1181/2352] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:03:29.078505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:29.090544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672c746-4677-4b39-b14f-ca47f2c4755d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:29.096177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:29.097188 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:29.097477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:29.097768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] PATH_INFO: `/v3/projects/11bf10102c9244f99f64ad5b4177407a/users/a84ea9c601c44d75a19dee660c4ca756/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:29.118881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11bf10102c9244f99f64ad5b4177407a, user_id=a84ea9c601c44d75a19dee660c4ca756, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:29.123382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:29.143731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1172/2353] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:29 2022] PUT /identity/v3/projects/11bf10102c9244f99f64ad5b4177407a/users/a84ea9c601c44d75a19dee660c4ca756/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:29.153346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e8a7f-feb3-4d90-a26a-691247b25e6c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:29.153808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e8a7f-feb3-4d90-a26a-691247b25e6c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:29.154145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e8a7f-feb3-4d90-a26a-691247b25e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:29.182820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-317e8a7f-feb3-4d90-a26a-691247b25e6c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:29.184133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-317e8a7f-feb3-4d90-a26a-691247b25e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:29.265875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1182/2354] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:29.277681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:29.279435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:29.328226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:29.343882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a223b4-adf3-4bfe-9bb0-ee16a93e58f3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:29.347733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ceeab-c4b5-4720-987c-4a51bdf9042d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:29.348441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ceeab-c4b5-4720-987c-4a51bdf9042d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:29.348705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ceeab-c4b5-4720-987c-4a51bdf9042d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:29.348989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ceeab-c4b5-4720-987c-4a51bdf9042d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:29.359986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ceeab-c4b5-4720-987c-4a51bdf9042d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:29.361510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ceeab-c4b5-4720-987c-4a51bdf9042d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:29.385362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1173/2355] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:03:32.970300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:32.970952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:32.971347 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:33.001831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] MFA Rules not processed for user `b86c21e37e2a49e9bdd48e15b091ce09`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:33.004591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:33.134148 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1183/2356] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:35.776362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:35.777177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:35.777177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:35.814602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] MFA Rules not processed for user `a84ea9c601c44d75a19dee660c4ca756`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:35.819550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:35.903701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1174/2357] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:35.908942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:35.929853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92798f5-24f6-416c-aa67-f2fca8c535ea None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:35.940948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:35.942095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:35.942949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:35.943642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:35.945737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:35.951036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:35.997641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1184/2358] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:03:35 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:03:36.001971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:36.013813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4fe17d-681f-4fd6-a1a4-fcbd9de8e4e1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:36.108222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:36.109300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:36.109869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:36.110435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:36.111631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:36.115090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:36.147631 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1175/2359] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:03:36 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:36.148988 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:36.165097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda108-2a6a-46bc-bd4d-fe4de676325a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:36.171985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:36.172738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:36.173015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:36.173503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:36.174253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:36.175901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:36.189309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1185/2360] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:36.194578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:36.213828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09b811-aeb0-4cf2-85ca-078e2df13d3c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:36.219772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:36.220846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:36.221215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:36.221660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] PATH_INFO: `/v3/projects/acceaf20f25848cbaeac142e0fc3607d/users/c4bbcc4d4ef14a0b8e1637846dadc9f2/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:36.245933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] RBAC: Authorizing `identity:create_grant(project_id=acceaf20f25848cbaeac142e0fc3607d, user_id=c4bbcc4d4ef14a0b8e1637846dadc9f2, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:36.249443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:36.266606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1176/2361] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:36 2022] PUT /identity/v3/projects/acceaf20f25848cbaeac142e0fc3607d/users/c4bbcc4d4ef14a0b8e1637846dadc9f2/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:03:36.270627 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:36.288925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d053f3b-7f7e-47f0-bb3c-275c2d250369 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:36.384326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:36.428013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:36.428747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:36.429078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:36.429299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:36.430051 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:36.433447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:36.452951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1186/2362] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:36.460583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:36.472377 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1c0ac-ead5-4a9f-bf8c-e03de87d749e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:36.479016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088cf1-0c2a-417d-aab1-07d57f9697e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:36.480858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088cf1-0c2a-417d-aab1-07d57f9697e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:36.481144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088cf1-0c2a-417d-aab1-07d57f9697e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:36.481422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088cf1-0c2a-417d-aab1-07d57f9697e6 admin admin] PATH_INFO: `/v3/projects/acceaf20f25848cbaeac142e0fc3607d/users/c4bbcc4d4ef14a0b8e1637846dadc9f2/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:36.483809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088cf1-0c2a-417d-aab1-07d57f9697e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acceaf20f25848cbaeac142e0fc3607d, user_id=c4bbcc4d4ef14a0b8e1637846dadc9f2, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:36.487225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088cf1-0c2a-417d-aab1-07d57f9697e6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:36.501593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1177/2363] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:36 2022] PUT /identity/v3/projects/acceaf20f25848cbaeac142e0fc3607d/users/c4bbcc4d4ef14a0b8e1637846dadc9f2/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:41.484597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:41.487365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:41.504481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31e5fb-e142-425e-a3ac-f13e2274de07 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:41.621851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:41.622816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:41.623285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:41.623744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] PATH_INFO: `/v3/users/06fa8faf31264fdda109b3af3e9e080f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:41.625241 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06fa8faf31264fdda109b3af3e9e080f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:41.627784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:41.716006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06fa8faf31264fdda109b3af3e9e080f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:41.738182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06fa8faf31264fdda109b3af3e9e080f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:41.751250 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06fa8faf31264fdda109b3af3e9e080f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:41.751877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06fa8faf31264fdda109b3af3e9e080f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:41.752972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06fa8faf31264fdda109b3af3e9e080f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:41.772995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1187/2364] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:41 2022] DELETE /identity/v3/users/06fa8faf31264fdda109b3af3e9e080f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:41.777448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:41.778062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:41.778062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:41.797968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:41.799268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:41.832455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:41.905407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1178/2365] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:41.925942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:41.927489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:42.039442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d092874-3b09-4cf4-86cf-dee62d84040b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:42.065954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d4b2e-14ff-4ec1-879e-01b27d7ff7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:42.065954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d4b2e-14ff-4ec1-879e-01b27d7ff7bf service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:42.066329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d4b2e-14ff-4ec1-879e-01b27d7ff7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:42.066971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d4b2e-14ff-4ec1-879e-01b27d7ff7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:42.084338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d4b2e-14ff-4ec1-879e-01b27d7ff7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:42.086045 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d4b2e-14ff-4ec1-879e-01b27d7ff7bf service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:42.092810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:42.101041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1188/2366] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:42.587131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:42.590531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:42.607069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94317da7-fb9a-4ea6-9678-2a9d21c965ab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:42.613468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:42.614484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:42.615165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:42.615462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] PATH_INFO: `/v3/projects/8969431739ca45c4a0adcaf6f4891c3d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:42.616770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8969431739ca45c4a0adcaf6f4891c3d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:42.618854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:42.695010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8969431739ca45c4a0adcaf6f4891c3d', 'request_id': 'req-61b15644-5bed-48bd-841a-4101156858cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:42.706141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8969431739ca45c4a0adcaf6f4891c3d', 'request_id': 'req-61b15644-5bed-48bd-841a-4101156858cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:42.718082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8969431739ca45c4a0adcaf6f4891c3d', 'request_id': 'req-61b15644-5bed-48bd-841a-4101156858cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:42.745505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] The token cache is being invalidate because project 8969431739ca45c4a0adcaf6f4891c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:42.745943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:42.746626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61b15644-5bed-48bd-841a-4101156858cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:42.748595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1179/2367] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:42 2022] DELETE /identity/v3/projects/8969431739ca45c4a0adcaf6f4891c3d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:42.753381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:42.753920 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:42.754408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:42.774940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:42.776817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:42.855240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1189/2368] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:42.863834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:42.864810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:42.865083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:42.888306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:42.889657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:42.899672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1180/2369] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:42.904986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:42.916825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e2bdf-2265-4190-aecb-39f1be548da1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:42.925562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:42.926740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:42.927036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:42.927316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:42.928213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:03:42.928560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:42.930826 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:42.951632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1190/2370] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:03:42.956602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:42.973884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d154c-4c19-4fc1-82f6-cf8e914ed214 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:42.979868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:42.981528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:42.981991 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:42.983560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:42.984501 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:42.986786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:43.029713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1181/2371] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:03:42 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:43.031934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:43.054049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fff4b-e961-4b26-abef-399d122eee02 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:43.152904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:43.153604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:43.153888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:43.154304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:43.155035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:43.157064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:43.186373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1191/2372] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:03:43 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:03:43.191387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:43.203305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4b353-ed00-494e-8b45-0fd6e64c72d3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:43.208759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:43.209476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:43.209995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:43.210276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:43.211023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:43.212799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:43.231630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1182/2373] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:03:43.232018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:43.247107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae246ffe-3269-44dc-9c61-2aedc3d374ac admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:43.269439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804bc40-3b22-4fe1-8c08-9d1cf9cae2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:43.270509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804bc40-3b22-4fe1-8c08-9d1cf9cae2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:43.271784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804bc40-3b22-4fe1-8c08-9d1cf9cae2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:43.275207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804bc40-3b22-4fe1-8c08-9d1cf9cae2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:43.276832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804bc40-3b22-4fe1-8c08-9d1cf9cae2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:43.280689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804bc40-3b22-4fe1-8c08-9d1cf9cae2f9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:43.302424 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1192/2374] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:03:43.307393 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:43.321918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda0fe1-82ad-4898-aca0-c83574317d0e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:43.332061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:43.333395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:43.333877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:43.334296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] PATH_INFO: `/v3/projects/acdb252455a749c1ae228292ae29f2c5/users/65fb1d6ea8744adf8cbde6bf86569bfd/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:43.354457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdb252455a749c1ae228292ae29f2c5, user_id=65fb1d6ea8744adf8cbde6bf86569bfd, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:43.357589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:43.386169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1183/2375] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:03:43 2022] PUT /identity/v3/projects/acdb252455a749c1ae228292ae29f2c5/users/65fb1d6ea8744adf8cbde6bf86569bfd/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:43.390596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997af7e-f5a9-4a95-b09e-39d1e7958a68 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:43.390894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997af7e-f5a9-4a95-b09e-39d1e7958a68 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:43.391182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997af7e-f5a9-4a95-b09e-39d1e7958a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:43.421637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e997af7e-f5a9-4a95-b09e-39d1e7958a68 None None] MFA Rules not processed for user `65fb1d6ea8744adf8cbde6bf86569bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:43.423156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e997af7e-f5a9-4a95-b09e-39d1e7958a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:43.516697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:43.534900 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1193/2376] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:43.547627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:43.549563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:43.610194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:43.616874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb6e0e-d75d-4580-824d-ea5e0cb770a1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:43.624524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:43.625505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:43.625791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:43.626091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:43.639135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:43.640882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:43.650747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1184/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:44.135307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c297d-c940-44ec-ac20-20d1991fba33 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:44.136087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c297d-c940-44ec-ac20-20d1991fba33 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:44.136087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c297d-c940-44ec-ac20-20d1991fba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:44.193431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-712c297d-c940-44ec-ac20-20d1991fba33 None None] MFA Rules not processed for user `c4bbcc4d4ef14a0b8e1637846dadc9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:44.194737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-712c297d-c940-44ec-ac20-20d1991fba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:44.349454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1194/2378] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:44.374723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:44.378217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:44.492789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b7c1-00ab-4ea4-a7c9-53a80476677f service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:44.499777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:44.501161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:44.501777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:44.502457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:44.505937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:44.611224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:44.614590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:44.631169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1185/2379] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:44.789421 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f19a-670c-4876-8eb2-0d3857364174 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:44.789817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f19a-670c-4876-8eb2-0d3857364174 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:44.790173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f19a-670c-4876-8eb2-0d3857364174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:44.810229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a537f19a-670c-4876-8eb2-0d3857364174 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:44.811844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a537f19a-670c-4876-8eb2-0d3857364174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:44.884651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1195/2380] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:03:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:44.932598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:44.934068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:45.011233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838f1f-1242-4d5c-802f-6a03adc9c0f1 service glance] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:45.017703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:45.018364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:45.018650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:45.018952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:45.029478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:45.031603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:45.042825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1186/2381] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:03:50.446386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:50.448896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:50.449032 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:50.482252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:50.502188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:50.518774 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1196/2382] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:03:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:03:50.535134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:50.555730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b7e88-12ad-45e9-a0c0-94119ab3f6ce service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:50.568467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:50.571333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:50.571766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:50.572925 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:50.602485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:50.604166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:50.616130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1187/2383] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:03:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:03:57.305749 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:57.308682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:57.513192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae1419-412b-4626-bb74-d41459eae40b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:57.522426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:57.525181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:57.526256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:57.526730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:57.530287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:57.558577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:57.562602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635a663-9ee1-481d-acfe-f189f951a30b service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:57.586192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1197/2384] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:03:58.355341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:58.399951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:58.467811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58135d1b-7c22-432f-81be-f0d98fa5e939 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:58.474697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:58.475762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:58.476312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:58.476468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] PATH_INFO: `/v3/users/65fb1d6ea8744adf8cbde6bf86569bfd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:58.480212 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65fb1d6ea8744adf8cbde6bf86569bfd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:58.496470 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:58.593101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65fb1d6ea8744adf8cbde6bf86569bfd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:58.623448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65fb1d6ea8744adf8cbde6bf86569bfd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:58.638031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65fb1d6ea8744adf8cbde6bf86569bfd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:58.660914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65fb1d6ea8744adf8cbde6bf86569bfd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:58.661615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65fb1d6ea8744adf8cbde6bf86569bfd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:58.680053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1188/2385] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:03:58 2022] DELETE /identity/v3/users/65fb1d6ea8744adf8cbde6bf86569bfd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:03:58.684382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175b4dc-f132-418a-9bdf-65c547e897ff None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:58.685705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175b4dc-f132-418a-9bdf-65c547e897ff None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:58.686403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175b4dc-f132-418a-9bdf-65c547e897ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:58.737300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8175b4dc-f132-418a-9bdf-65c547e897ff None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:58.738937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8175b4dc-f132-418a-9bdf-65c547e897ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:58.745650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:58.892862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1198/2386] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:58.904700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:58.907590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:58.935670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61e522-ad9c-4e89-ba02-917fc7f5e416 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:58.935963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61e522-ad9c-4e89-ba02-917fc7f5e416 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:58.936307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61e522-ad9c-4e89-ba02-917fc7f5e416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:58.957819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8a61e522-ad9c-4e89-ba02-917fc7f5e416 None None] MFA Rules not processed for user `a84ea9c601c44d75a19dee660c4ca756`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:03:58.959254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8a61e522-ad9c-4e89-ba02-917fc7f5e416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:58.995894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f9e9-f6b5-4b2b-8b09-6cec2d79e098 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:59.000047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:59.000751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:59.001035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:59.001305 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:59.014645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:59.016333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:59.028189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1189/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:03:59.030717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1199/2388] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:03:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:03:59.575565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:03:59.578771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:03:59.611225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237d8a8-8024-49e3-9f16-4c07966e93c7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:03:59.625909 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:03:59.626608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:03:59.627614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:03:59.627614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] PATH_INFO: `/v3/projects/acdb252455a749c1ae228292ae29f2c5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:03:59.631471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=acdb252455a749c1ae228292ae29f2c5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:03:59.634923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:03:59.699896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:03:59.723453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acdb252455a749c1ae228292ae29f2c5', 'request_id': 'req-f7e5c719-1eb5-434c-b6da-3930beb1da6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:59.732889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acdb252455a749c1ae228292ae29f2c5', 'request_id': 'req-f7e5c719-1eb5-434c-b6da-3930beb1da6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:59.745265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acdb252455a749c1ae228292ae29f2c5', 'request_id': 'req-f7e5c719-1eb5-434c-b6da-3930beb1da6e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:59.766173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] The token cache is being invalidate because project acdb252455a749c1ae228292ae29f2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:03:59.766734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:59.767593 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:03:59.769603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1190/2389] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:03:59 2022] DELETE /identity/v3/projects/acdb252455a749c1ae228292ae29f2c5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:04:01.727249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c28a-f11b-42b6-973c-54a26e39d17b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:01.727797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c28a-f11b-42b6-973c-54a26e39d17b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:01.727797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c28a-f11b-42b6-973c-54a26e39d17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:01.749115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0e87c28a-f11b-42b6-973c-54a26e39d17b None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:04:01.751876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0e87c28a-f11b-42b6-973c-54a26e39d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:01.847037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1200/2390] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:04:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:04:01.935256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:01.937190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:02.090677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c719-1eb5-434c-b6da-3930beb1da6e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:02.112085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8046588-6695-4872-8c14-aa1e022f999b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:02.113141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8046588-6695-4872-8c14-aa1e022f999b service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:02.113650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8046588-6695-4872-8c14-aa1e022f999b service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:02.114163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8046588-6695-4872-8c14-aa1e022f999b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:02.138149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8046588-6695-4872-8c14-aa1e022f999b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:02.139951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8046588-6695-4872-8c14-aa1e022f999b service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:02.190804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1191/2391] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:04:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:04:11.833692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=73 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:04:12.180902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=43,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:04:20.821885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574652f8-9f1d-44fd-904c-8576e86f818f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:20.823893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574652f8-9f1d-44fd-904c-8576e86f818f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:20.823893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574652f8-9f1d-44fd-904c-8576e86f818f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:20.824365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1201/2392] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:04:20.832892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30b30-dce7-482f-b0fa-35a29a16f08b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:20.833487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30b30-dce7-482f-b0fa-35a29a16f08b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:20.833487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30b30-dce7-482f-b0fa-35a29a16f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:20.853673 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2cb30b30-dce7-482f-b0fa-35a29a16f08b None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:04:20.855623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2cb30b30-dce7-482f-b0fa-35a29a16f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:20.942890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1192/2393] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:04:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:04:20.955331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574652f8-9f1d-44fd-904c-8576e86f818f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:20.956738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-574652f8-9f1d-44fd-904c-8576e86f818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:21.024042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574652f8-9f1d-44fd-904c-8576e86f818f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:21.028588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a6307-5b5b-410f-9b4d-7b89b25b452e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'deb87accdcab4759add4b76958d151a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:21.029313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a6307-5b5b-410f-9b4d-7b89b25b452e service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.029540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a6307-5b5b-410f-9b4d-7b89b25b452e service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.029819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a6307-5b5b-410f-9b4d-7b89b25b452e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.040733 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a6307-5b5b-410f-9b4d-7b89b25b452e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:21.042212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a6307-5b5b-410f-9b4d-7b89b25b452e service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:21.055138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1202/2394] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:04:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:04:21.123434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b4ff8-80ca-4c1a-bf07-5396681d1a83 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.123728 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b4ff8-80ca-4c1a-bf07-5396681d1a83 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.124016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b4ff8-80ca-4c1a-bf07-5396681d1a83 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.125450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1193/2395] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:04:21.130340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47a119-ca60-4588-b044-788483d17612 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.130628 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47a119-ca60-4588-b044-788483d17612 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.130910 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47a119-ca60-4588-b044-788483d17612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.150233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3d47a119-ca60-4588-b044-788483d17612 None None] MFA Rules not processed for user `393c4bea16074727958ea40ab497948d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:04:21.152063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3d47a119-ca60-4588-b044-788483d17612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:21.204218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1203/2396] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:04:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:04:21.216713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b4ff8-80ca-4c1a-bf07-5396681d1a83 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:21.218840 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf9b4ff8-80ca-4c1a-bf07-5396681d1a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:21.231693 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b4ff8-80ca-4c1a-bf07-5396681d1a83 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:21.235662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce1385-74e5-4a6f-84af-e3e2f7014a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:21.236301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce1385-74e5-4a6f-84af-e3e2f7014a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.236605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce1385-74e5-4a6f-84af-e3e2f7014a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.236875 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce1385-74e5-4a6f-84af-e3e2f7014a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.244016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce1385-74e5-4a6f-84af-e3e2f7014a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:21.246107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce1385-74e5-4a6f-84af-e3e2f7014a79 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:21.254557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1194/2397] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:04:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:04:21.369861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aeffa1-3212-42ad-8a91-2be752aaf7f9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.370153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aeffa1-3212-42ad-8a91-2be752aaf7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.370964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aeffa1-3212-42ad-8a91-2be752aaf7f9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.372348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1204/2398] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jul 15 14:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 14:04:21.378058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504be2-9d0d-4500-9cd3-a095f874d8ef None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.378353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504be2-9d0d-4500-9cd3-a095f874d8ef None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.378636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504be2-9d0d-4500-9cd3-a095f874d8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.397371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1b504be2-9d0d-4500-9cd3-a095f874d8ef None None] MFA Rules not processed for user `e424ea6d03414857bc408e49b8df5429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:04:21.398615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1b504be2-9d0d-4500-9cd3-a095f874d8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:21.408228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1195/2399] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jul 15 14:04:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:04:21.419302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aeffa1-3212-42ad-8a91-2be752aaf7f9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:21.421617 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-99aeffa1-3212-42ad-8a91-2be752aaf7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:21.436530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aeffa1-3212-42ad-8a91-2be752aaf7f9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:21.441622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:21.442730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:21.443114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:21.443401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:21.455802 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:21.457386 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:21.470195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1205/2400] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:04:23.734050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be524afb-4749-41a8-be16-83942a09da03 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:23.734592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be524afb-4749-41a8-be16-83942a09da03 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:23.734766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be524afb-4749-41a8-be16-83942a09da03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:23.754590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-be524afb-4749-41a8-be16-83942a09da03 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:04:23.755814 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-be524afb-4749-41a8-be16-83942a09da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:23.763187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1196/2401] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:04:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:04:23.775625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:23.787585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08e67c-cfe0-493d-918b-c52d139bb0bb service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:23.794207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ea257-1de0-475b-a361-2b253a9f7066 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:23.794876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea257-1de0-475b-a361-2b253a9f7066 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:23.795158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea257-1de0-475b-a361-2b253a9f7066 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:23.795440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea257-1de0-475b-a361-2b253a9f7066 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:23.804648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ea257-1de0-475b-a361-2b253a9f7066 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:23.806257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ea257-1de0-475b-a361-2b253a9f7066 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:23.815437 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1206/2402] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:04:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:04:33.748851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=43 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:04:33.811659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=61 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:04:41.169468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be524afb-4749-41a8-be16-83942a09da03 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:41.179374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be524afb-4749-41a8-be16-83942a09da03 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:41.183262 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:41.184005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:41.184419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:41.184764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:41.187079 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:41.263672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:41.265222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:41.277378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1197/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:04:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:04:46.301121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fef59-fe77-4db3-ba91-ed006e33e4b5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:46.301657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fef59-fe77-4db3-ba91-ed006e33e4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:46.301768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fef59-fe77-4db3-ba91-ed006e33e4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:46.320459 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-005fef59-fe77-4db3-ba91-ed006e33e4b5 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:04:46.321680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-005fef59-fe77-4db3-ba91-ed006e33e4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:04:46.330380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1207/2404] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jul 15 14:04:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:04:46.354275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:04:46.366758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4268562-df5a-4b37-9675-026ec0876e48 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:04:46.372310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f239f9-0bd8-4b08-b04a-ce81b0a19880 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:04:46.373040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f239f9-0bd8-4b08-b04a-ce81b0a19880 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:04:46.373326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f239f9-0bd8-4b08-b04a-ce81b0a19880 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:04:46.373604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f239f9-0bd8-4b08-b04a-ce81b0a19880 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:04:46.381997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f239f9-0bd8-4b08-b04a-ce81b0a19880 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:04:46.383583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f239f9-0bd8-4b08-b04a-ce81b0a19880 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:04:46.391588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1198/2405] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:04:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:04:56.314867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:04:56.388437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=36 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:05:08.118044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39402fc-9cbb-4748-9c83-88b8a1f5d111 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:05:08.118639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39402fc-9cbb-4748-9c83-88b8a1f5d111 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:05:08.118750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39402fc-9cbb-4748-9c83-88b8a1f5d111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:05:08.137578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a39402fc-9cbb-4748-9c83-88b8a1f5d111 None None] MFA Rules not processed for user `b86c21e37e2a49e9bdd48e15b091ce09`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:05:08.138851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a39402fc-9cbb-4748-9c83-88b8a1f5d111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:05:08.197026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1208/2406] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:05:18.189561 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=25 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:05:38.566504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:05:38.567157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:05:38.567157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:05:38.589124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] MFA Rules not processed for user `520583af1f264f22a86722b0cb9ff405`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:05:38.591380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:05:38.681010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1199/2407] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:05:39.524758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:05:39.525361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:05:39.525361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:05:39.549648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] MFA Rules not processed for user `96d4c389c4da4d2188e8a32e5ed1be7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:05:39.551694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:05:39.638456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1209/2408] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:05:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:05:48.669607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=25 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:05:49.624839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=25 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:05:53.923258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:05:53.941833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18173aa-74ba-41f2-834e-f1e595b6e972 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:05:53.947670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:05:53.948836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:05:53.949347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:05:53.949844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:05:53.953504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:05:53.973877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:05:53.976729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:05:53.999654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1200/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:05:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:00.333608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:00.345273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeecdab-98c2-4bec-87ab-8d5f60263c75 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:00.352185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:00.353359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:00.353868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:00.354377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:00.358256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:00.384256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:00.387223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:00.408557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1210/2410] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:03.023056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:03.033312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d36c07-b186-4273-a0f0-adf6ec65f636 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:03.037113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:03.037779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:03.038068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:03.038344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:03.040417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:03.053070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:03.054529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:03.068982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1201/2411] 10.222.0.16 () {60 vars in 1396 bytes} [Fri Jul 15 14:06:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:08.002333 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:08.014571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76163c0-3ebf-4296-b77c-3321c8093265 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:08.018592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:08.019263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:08.019546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:08.019831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:08.021732 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:08.097789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:08.099255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:08.111381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1211/2412] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:12.010551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:12.011997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:12.030660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370564d-8854-4955-95e6-28522fb8360e service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:12.036972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:12.037862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.038149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.038445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] PATH_INFO: `/v3/users/b86c21e37e2a49e9bdd48e15b091ce09` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.039676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86c21e37e2a49e9bdd48e15b091ce09)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:12.041665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:12.094062 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86c21e37e2a49e9bdd48e15b091ce09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.109945 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86c21e37e2a49e9bdd48e15b091ce09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.124561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86c21e37e2a49e9bdd48e15b091ce09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.137133 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86c21e37e2a49e9bdd48e15b091ce09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.138775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d3eb1be1-6fef-402a-ba4c-2029274c5ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86c21e37e2a49e9bdd48e15b091ce09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.151359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1202/2413] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:12 2022] DELETE /identity/v3/users/b86c21e37e2a49e9bdd48e15b091ce09 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:12.380052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:12.399248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcf3d9-9a9c-4472-a10e-6ca50ef1a768 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:12.472100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:12.473127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.473489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.473857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] PATH_INFO: `/v3/projects/77c8c864a31b402880c334c39fbe5a34` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.475273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c8c864a31b402880c334c39fbe5a34)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:12.477645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:12.544295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c8c864a31b402880c334c39fbe5a34', 'request_id': 'req-0bd47e95-a362-449e-b0dc-ea04c6631b14'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.555950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c8c864a31b402880c334c39fbe5a34', 'request_id': 'req-0bd47e95-a362-449e-b0dc-ea04c6631b14'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.574593 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c8c864a31b402880c334c39fbe5a34', 'request_id': 'req-0bd47e95-a362-449e-b0dc-ea04c6631b14'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.587683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] The token cache is being invalidate because project 77c8c864a31b402880c334c39fbe5a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:12.588066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.588820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0bd47e95-a362-449e-b0dc-ea04c6631b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:12.590442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1212/2414] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:12 2022] DELETE /identity/v3/projects/77c8c864a31b402880c334c39fbe5a34 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:12.596433 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.596855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.597177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.617886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:12.619138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:12.689583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1203/2415] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:12.698225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.698505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.698824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.724224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:12.725490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:12.733675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1213/2416] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:12.739279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:12.751023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed0903-c259-4686-b3f3-49e9f7ecffe1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:12.755737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:12.756474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.756767 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.757048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.757809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:06:12.758116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:12.760067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:12.774860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1204/2417] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:06:12.780976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:12.787085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:12.791804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32597b6d-405b-4153-b5d3-3cf718309ba4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:12.799334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:12.800434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.800970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.801382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.802452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:12.804782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:12.837034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1214/2418] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:12 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:12.838496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:12.848848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffe971-e7b8-4a47-b7a8-e37cb4fa065b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:12.879865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:12.915601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:12.916293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.916601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.916868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.917606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:12.920628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:12.948849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1205/2419] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:12 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:12.952977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:12.966442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07decd2-015c-4b4a-8439-25093230b73b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:12.973756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9e841-739c-44d8-9e75-41fae9b42b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:12.974544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9e841-739c-44d8-9e75-41fae9b42b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:12.974929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9e841-739c-44d8-9e75-41fae9b42b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:12.976083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9e841-739c-44d8-9e75-41fae9b42b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:12.976083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9e841-739c-44d8-9e75-41fae9b42b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:12.977889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9e841-739c-44d8-9e75-41fae9b42b04 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:12.991452 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1215/2420] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:12.995481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.006921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b947e-5a30-4336-9247-3927d46cdbc4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.008694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.008985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.009265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.012361 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.013135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.013419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.014351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.015117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.016774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.027718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1206/2421] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:06:13.027924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:13.029083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:13.031637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.041677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa37bf-8cf2-4420-be42-031812f4c456 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.046962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322009f-a037-4d06-8e4a-3134bac3be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.047841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322009f-a037-4d06-8e4a-3134bac3be3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.048121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322009f-a037-4d06-8e4a-3134bac3be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.048427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322009f-a037-4d06-8e4a-3134bac3be3b admin admin] PATH_INFO: `/v3/projects/8b4e29adf3b84ba99abb9c6473a69d35/users/6d238c742ddb4769b88f9966681cf6f2/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.066392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322009f-a037-4d06-8e4a-3134bac3be3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4e29adf3b84ba99abb9c6473a69d35, user_id=6d238c742ddb4769b88f9966681cf6f2, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.069108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322009f-a037-4d06-8e4a-3134bac3be3b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.083601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1207/2422] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:13 2022] PUT /identity/v3/projects/8b4e29adf3b84ba99abb9c6473a69d35/users/6d238c742ddb4769b88f9966681cf6f2/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:13.086508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1216/2423] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jul 15 14:06:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:06:13.094886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.094886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.095422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.125005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] MFA Rules not processed for user `6d238c742ddb4769b88f9966681cf6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:13.126286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:13.135470 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.137035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:13.197391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1208/2424] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:13.201654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.215745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce29e2-e115-4348-989c-fa2eda98eb9a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.266268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe53366-6323-4ee6-bd6b-e13b016733a1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.271082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.271741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.272092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.272695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.273753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.275739 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.283745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.284491 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.284804 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.285116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.307564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1209/2425] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:13 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:13.307842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:13.308607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.313605 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:13.362215 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.368741 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.379279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1217/2426] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:06:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 14:06:13.379482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdace066-f405-4ef1-9958-d99d48d03bad admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.385698 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.386479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.386769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.387049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.387785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.390500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.417222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1210/2427] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:13 2022] POST /identity/v3/users => generated 308 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:13.423220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.435942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426a110-cca7-4987-9145-86c5e5b54228 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.440792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.441578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.441869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.442151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.443549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.445192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.456556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1218/2428] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:13.461560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.475677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0da117-5aa2-4265-99c2-ea60bb258f2e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.483651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.487944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.488364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.488740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] PATH_INFO: `/v3/projects/067e58ec3817430eb31313345c2d0062/users/20cd3466566a454bb8ab1f334881e366/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.506671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] RBAC: Authorizing `identity:create_grant(project_id=067e58ec3817430eb31313345c2d0062, user_id=20cd3466566a454bb8ab1f334881e366, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.509315 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.524621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1211/2429] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:13 2022] PUT /identity/v3/projects/067e58ec3817430eb31313345c2d0062/users/20cd3466566a454bb8ab1f334881e366/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:13.529546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.540586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a4df1-097f-4ae6-83eb-b8311d944931 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.614256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ca69e-09fd-4abb-bf8b-aa8d803788e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.614936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ca69e-09fd-4abb-bf8b-aa8d803788e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.615223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ca69e-09fd-4abb-bf8b-aa8d803788e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.615498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ca69e-09fd-4abb-bf8b-aa8d803788e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.616334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ca69e-09fd-4abb-bf8b-aa8d803788e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.618473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ca69e-09fd-4abb-bf8b-aa8d803788e2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.629939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1219/2430] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:13.641386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.652568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565413b-2f2e-465c-b33c-c334323c940c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.659861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.660988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.661364 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.661815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] PATH_INFO: `/v3/projects/067e58ec3817430eb31313345c2d0062/users/20cd3466566a454bb8ab1f334881e366/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.663997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=067e58ec3817430eb31313345c2d0062, user_id=20cd3466566a454bb8ab1f334881e366, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.666738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.678524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1212/2431] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:13 2022] PUT /identity/v3/projects/067e58ec3817430eb31313345c2d0062/users/20cd3466566a454bb8ab1f334881e366/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:13.688888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.689277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.689649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.720915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] MFA Rules not processed for user `20cd3466566a454bb8ab1f334881e366`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:13.722186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:13.823530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1220/2432] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:13.837334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:13.839192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:13.912299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5352411-fe59-4447-825b-c71e0371e1ae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:13.916103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc7195-e63f-46d2-856f-51da366727a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:13.916819 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc7195-e63f-46d2-856f-51da366727a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:13.917101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc7195-e63f-46d2-856f-51da366727a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:13.917381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc7195-e63f-46d2-856f-51da366727a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:13.927798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc7195-e63f-46d2-856f-51da366727a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:13.929304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc7195-e63f-46d2-856f-51da366727a4 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:13.939867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1213/2433] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:14.088887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:14.127507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:14.162997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd08fa4-4a4e-4079-919a-58bd3db56a0b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:14.167978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:14.169066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:14.169339 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:14.169608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] PATH_INFO: `/v3/users/6d238c742ddb4769b88f9966681cf6f2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:14.170846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d238c742ddb4769b88f9966681cf6f2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:14.172877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:14.232611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d238c742ddb4769b88f9966681cf6f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.252058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d238c742ddb4769b88f9966681cf6f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.261266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d238c742ddb4769b88f9966681cf6f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.261978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d238c742ddb4769b88f9966681cf6f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.262637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d238c742ddb4769b88f9966681cf6f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.279942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1221/2434] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:14 2022] DELETE /identity/v3/users/6d238c742ddb4769b88f9966681cf6f2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:14.284860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:14.285477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:14.285477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:14.304170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:14.305701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:14.366805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:14.378887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1214/2435] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:14.389059 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:14.390643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:14.465273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e13f4-585c-4f57-8646-73f7f3d2cd54 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:14.470217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:14.470977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:14.471330 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:14.471613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:14.483392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:14.485284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:14.496489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1222/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:06:14.803059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:14.804907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:14.844656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77cb94-474a-4974-8d01-e55eb3232cd4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:14.850874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:14.852747 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:14.853455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:14.854149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] PATH_INFO: `/v3/projects/8b4e29adf3b84ba99abb9c6473a69d35` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:14.856482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4e29adf3b84ba99abb9c6473a69d35)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:14.861257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:14.944372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4e29adf3b84ba99abb9c6473a69d35', 'request_id': 'req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.959994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4e29adf3b84ba99abb9c6473a69d35', 'request_id': 'req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.971689 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4e29adf3b84ba99abb9c6473a69d35', 'request_id': 'req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.990826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] The token cache is being invalidate because project 8b4e29adf3b84ba99abb9c6473a69d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:14.991373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.992632 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:14.995654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1215/2437] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:14 2022] DELETE /identity/v3/projects/8b4e29adf3b84ba99abb9c6473a69d35 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:15.002019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:15.012944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0332a-c824-4c79-a1c2-ee10d190096e service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:15.076706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:15.095240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:15.096958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:15.097477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:15.097996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] PATH_INFO: `/v3/users/20cd3466566a454bb8ab1f334881e366` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:15.100206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=20cd3466566a454bb8ab1f334881e366)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:15.104970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:15.170756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20cd3466566a454bb8ab1f334881e366'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.183569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20cd3466566a454bb8ab1f334881e366'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.193850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20cd3466566a454bb8ab1f334881e366'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.194487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20cd3466566a454bb8ab1f334881e366'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.195092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac9ad024-2a70-4efc-8aaa-d47c0a71c3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20cd3466566a454bb8ab1f334881e366'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.209789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1223/2438] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:15 2022] DELETE /identity/v3/users/20cd3466566a454bb8ab1f334881e366 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:15.533529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:15.537646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:15.559152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3bfd-c786-41a0-b175-c80c2ea0fa9e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:15.628072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:15.629151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:15.629505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:15.629851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] PATH_INFO: `/v3/projects/067e58ec3817430eb31313345c2d0062` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:15.631194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=067e58ec3817430eb31313345c2d0062)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:15.633427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:15.694924 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067e58ec3817430eb31313345c2d0062', 'request_id': 'req-ead5096e-05be-4e51-87a3-5e5065d2a5cc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.705215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067e58ec3817430eb31313345c2d0062', 'request_id': 'req-ead5096e-05be-4e51-87a3-5e5065d2a5cc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.716011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067e58ec3817430eb31313345c2d0062', 'request_id': 'req-ead5096e-05be-4e51-87a3-5e5065d2a5cc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.730525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] The token cache is being invalidate because project 067e58ec3817430eb31313345c2d0062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:15.730822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.731487 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ead5096e-05be-4e51-87a3-5e5065d2a5cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:15.733588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1216/2439] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:15 2022] DELETE /identity/v3/projects/067e58ec3817430eb31313345c2d0062 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:15.741015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:15.741580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:15.741832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:15.761310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:15.762700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:15.830151 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1224/2440] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:15.837995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:15.838378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:15.838731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:15.860324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:15.861687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:15.871397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1217/2441] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:15.877237 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:15.889566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2476660-6c0c-4cb4-8339-11210c34ba38 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:15.897612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:15.898943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:15.899334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:15.899801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:15.900700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:06:15.901074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:15.902933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:15.916835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1225/2442] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:06:15.922243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:15.928112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:15.933345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c981345-d5ed-4327-96ad-a67b13d67980 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:15.942330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:15.943176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:15.943527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:15.944182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:15.945861 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:15.948153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:15.983705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1218/2443] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:15 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:15.985597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:15.999247 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6babfd-2c72-427e-bd03-69ae81266ac8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.111968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.112699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.112986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.113268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.114205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.116263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.127460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:16.164967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1226/2444] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:15 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:16.167230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.178848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d6944-530f-4263-a665-901d2e0dfc7e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.184172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.186184 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.186481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.186756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.187723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.189612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.203419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1219/2445] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:16.207500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.218618 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9579b-73d6-43c4-b34f-8e406a4714c1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.225903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdfeed-7e7d-4413-91a1-f37de740dfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.226769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdfeed-7e7d-4413-91a1-f37de740dfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.227221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdfeed-7e7d-4413-91a1-f37de740dfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.227504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdfeed-7e7d-4413-91a1-f37de740dfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.228274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdfeed-7e7d-4413-91a1-f37de740dfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.229971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdfeed-7e7d-4413-91a1-f37de740dfd0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.243048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1227/2446] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:16.248517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.259578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe12c51-f07d-424e-b266-879a71fb31ff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.265179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.266275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.266633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.267035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] PATH_INFO: `/v3/projects/09502e4f3852473da3f347ff95e2f75e/users/ad1269e7eb7940418aaf6cec632cd618/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.291156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09502e4f3852473da3f347ff95e2f75e, user_id=ad1269e7eb7940418aaf6cec632cd618, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.294001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.311598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1220/2447] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:16 2022] PUT /identity/v3/projects/09502e4f3852473da3f347ff95e2f75e/users/ad1269e7eb7940418aaf6cec632cd618/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:16.324216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.324663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.325191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.364365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] MFA Rules not processed for user `ad1269e7eb7940418aaf6cec632cd618`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:16.365649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:16.449894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1228/2448] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:16.455558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.533371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad51fb4-370b-4b67-aea4-c6f5817d2f7a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.538405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.539073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.539341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.539619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.540341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.542366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.577887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1221/2449] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:16 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:16.582884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.595220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38009cf0-b588-4ce6-8a53-dcc8e76326dc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.671100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.671889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.672251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.672775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.673299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.675263 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.706842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1229/2450] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:16 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:16.712011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.736986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3779d-8304-4107-9abd-5c3085f8dd68 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.750755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.752071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.752807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.753394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.755010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.758824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.775192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1222/2451] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:16.782154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.793794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c66ca0-4ace-4775-8f4e-529f08f528d5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.801118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.802302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.802912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.803347 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] PATH_INFO: `/v3/projects/71e9f04c57ac4e649d8dc96adb17caf8/users/34dddeec06ab4117b64aa84be2d28f77/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.824648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e9f04c57ac4e649d8dc96adb17caf8, user_id=34dddeec06ab4117b64aa84be2d28f77, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.828415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.850412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1230/2452] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:16 2022] PUT /identity/v3/projects/71e9f04c57ac4e649d8dc96adb17caf8/users/34dddeec06ab4117b64aa84be2d28f77/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:16.854317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.866763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb379b3-f880-4cc3-a0d1-e70fb2d210c9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.934366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293089a-72a7-4c50-8692-986d19f841cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.935135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293089a-72a7-4c50-8692-986d19f841cb admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.935424 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293089a-72a7-4c50-8692-986d19f841cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.935789 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293089a-72a7-4c50-8692-986d19f841cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.936703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293089a-72a7-4c50-8692-986d19f841cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.938236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293089a-72a7-4c50-8692-986d19f841cb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.949921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1223/2453] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:16.954551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:16.968102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01805f-b2e9-428c-84fe-9bd250cc0eed admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:16.973062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:16.973970 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:16.974251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:16.974524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] PATH_INFO: `/v3/projects/71e9f04c57ac4e649d8dc96adb17caf8/users/34dddeec06ab4117b64aa84be2d28f77/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:16.976566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e9f04c57ac4e649d8dc96adb17caf8, user_id=34dddeec06ab4117b64aa84be2d28f77, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:16.979286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:16.990938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1231/2454] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:16 2022] PUT /identity/v3/projects/71e9f04c57ac4e649d8dc96adb17caf8/users/34dddeec06ab4117b64aa84be2d28f77/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:17.000826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:17.001113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:17.001392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:17.040416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] MFA Rules not processed for user `34dddeec06ab4117b64aa84be2d28f77`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:17.040416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:17.045117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:17.152195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1224/2455] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:16 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:17.171933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:17.175320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:17.221037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:17.291989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b9b2d-582a-4fb2-8f09-b1da5b09ff10 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:17.299809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:17.301271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:17.302176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:17.302737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:17.324135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:17.327213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:17.350924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1232/2456] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:17.426118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:17.427858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:17.445185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b9f7b-fdeb-4cab-8d90-c37655fea549 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:17.449537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:17.450807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:17.451104 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:17.455883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:17.553719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:17.557052 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:17.571571 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1225/2457] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:20.999293 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:21.000938 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:21.097693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07b5c9-53c9-44bc-99ae-59da237f9497 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:21.102581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:21.103655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:21.103933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:21.104218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] PATH_INFO: `/v3/users/a84ea9c601c44d75a19dee660c4ca756` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:21.107048 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84ea9c601c44d75a19dee660c4ca756)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:21.109322 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:21.174154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84ea9c601c44d75a19dee660c4ca756'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.188783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84ea9c601c44d75a19dee660c4ca756'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.197551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84ea9c601c44d75a19dee660c4ca756'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.198647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84ea9c601c44d75a19dee660c4ca756'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.199411 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-df0c64df-e12c-46d5-add4-0ba020876151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84ea9c601c44d75a19dee660c4ca756'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.220901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1233/2458] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:20 2022] DELETE /identity/v3/users/a84ea9c601c44d75a19dee660c4ca756 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:21.321878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:21.333666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889eaff-77d2-4061-b5b2-88581a47474a service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:21.408834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:21.409741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:21.410026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:21.410308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] PATH_INFO: `/v3/users/ad1269e7eb7940418aaf6cec632cd618` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:21.411587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1269e7eb7940418aaf6cec632cd618)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:21.413703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:21.478266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:21.482190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1269e7eb7940418aaf6cec632cd618'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.499086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1269e7eb7940418aaf6cec632cd618'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.506901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1269e7eb7940418aaf6cec632cd618'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.518118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1269e7eb7940418aaf6cec632cd618'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.518823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1269e7eb7940418aaf6cec632cd618'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.526486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1226/2459] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:21 2022] DELETE /identity/v3/users/ad1269e7eb7940418aaf6cec632cd618 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:21.532719 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:21.533219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:21.533680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:21.553158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:21.554527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:21.598140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:21.625815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1234/2460] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:21.636557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:21.638459 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:21.679147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:21.681072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:21.701870 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419202d1-3605-4510-a5cf-1c280e94cfbf None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:21.707591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:21.708594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:21.708932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:21.709279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] PATH_INFO: `/v3/projects/11bf10102c9244f99f64ad5b4177407a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:21.711076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11bf10102c9244f99f64ad5b4177407a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:21.714334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35737dc7-cbeb-4882-b8ab-cc8e732207cf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:21.714579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:21.719600 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:21.720273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:21.720596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:21.720959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:21.735612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:21.737135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:21.748822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1227/2461] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:21.786483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11bf10102c9244f99f64ad5b4177407a', 'request_id': 'req-dc58a2b5-6efa-4d8a-ad21-0a48501af097'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.793453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11bf10102c9244f99f64ad5b4177407a', 'request_id': 'req-dc58a2b5-6efa-4d8a-ad21-0a48501af097'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.803914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11bf10102c9244f99f64ad5b4177407a', 'request_id': 'req-dc58a2b5-6efa-4d8a-ad21-0a48501af097'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.822923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] The token cache is being invalidate because project 11bf10102c9244f99f64ad5b4177407a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:21.823446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.824622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:21.826932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1235/2462] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:21 2022] DELETE /identity/v3/projects/11bf10102c9244f99f64ad5b4177407a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:21.833096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:21.848090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222ba6-2c5d-4fe6-9492-2ab70908a007 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:21.916167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:21.917182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:21.917504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:21.917810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] PATH_INFO: `/v3/users/c4bbcc4d4ef14a0b8e1637846dadc9f2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:21.919263 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4bbcc4d4ef14a0b8e1637846dadc9f2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:21.921350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:21.988898 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4bbcc4d4ef14a0b8e1637846dadc9f2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.003721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4bbcc4d4ef14a0b8e1637846dadc9f2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.005502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:22.014005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4bbcc4d4ef14a0b8e1637846dadc9f2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.024211 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4bbcc4d4ef14a0b8e1637846dadc9f2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.025825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4bbcc4d4ef14a0b8e1637846dadc9f2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.033796 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1228/2463] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:21 2022] DELETE /identity/v3/users/c4bbcc4d4ef14a0b8e1637846dadc9f2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:22.132428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:22.135142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:22.175548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58a2b5-6efa-4d8a-ad21-0a48501af097 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:22.247702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:22.248948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:22.249234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:22.249507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] PATH_INFO: `/v3/projects/09502e4f3852473da3f347ff95e2f75e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:22.250824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09502e4f3852473da3f347ff95e2f75e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:22.252759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:22.273436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:22.274893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:22.299453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8986c50-ea3e-4a86-843c-567184960b53 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:22.313904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09502e4f3852473da3f347ff95e2f75e', 'request_id': 'req-f2bcc212-d83a-42c7-aa36-7c08c64281a8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.321313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09502e4f3852473da3f347ff95e2f75e', 'request_id': 'req-f2bcc212-d83a-42c7-aa36-7c08c64281a8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.331054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09502e4f3852473da3f347ff95e2f75e', 'request_id': 'req-f2bcc212-d83a-42c7-aa36-7c08c64281a8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.339266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:22.352579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] The token cache is being invalidate because project 09502e4f3852473da3f347ff95e2f75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:22.353027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.354652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.357289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1236/2464] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:22 2022] DELETE /identity/v3/projects/09502e4f3852473da3f347ff95e2f75e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:22.363864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:22.365359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:22.381474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:22.382349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:22.382636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:22.382955 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] PATH_INFO: `/v3/projects/acceaf20f25848cbaeac142e0fc3607d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:22.384218 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acceaf20f25848cbaeac142e0fc3607d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:22.386185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:22.437041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc212-d83a-42c7-aa36-7c08c64281a8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:22.442750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acceaf20f25848cbaeac142e0fc3607d', 'request_id': 'req-2bd9675b-0368-4d39-8714-6d1ffbf331a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.449526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acceaf20f25848cbaeac142e0fc3607d', 'request_id': 'req-2bd9675b-0368-4d39-8714-6d1ffbf331a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.459251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acceaf20f25848cbaeac142e0fc3607d', 'request_id': 'req-2bd9675b-0368-4d39-8714-6d1ffbf331a6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.470864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] The token cache is being invalidate because project acceaf20f25848cbaeac142e0fc3607d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:22.471159 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.471849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.473474 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1229/2465] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:22 2022] DELETE /identity/v3/projects/acceaf20f25848cbaeac142e0fc3607d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:22.513881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:22.514796 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:22.515090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:22.515371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] PATH_INFO: `/v3/users/34dddeec06ab4117b64aa84be2d28f77` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:22.516837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] RBAC: Authorizing `identity:delete_user(user_id=34dddeec06ab4117b64aa84be2d28f77)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:22.518869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:22.585902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34dddeec06ab4117b64aa84be2d28f77'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.600835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34dddeec06ab4117b64aa84be2d28f77'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.609488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34dddeec06ab4117b64aa84be2d28f77'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.610130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34dddeec06ab4117b64aa84be2d28f77'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.610743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca87b824-8bb9-499d-a77e-feb5ca6492be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34dddeec06ab4117b64aa84be2d28f77'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:22.624584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1237/2466] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:22 2022] DELETE /identity/v3/users/34dddeec06ab4117b64aa84be2d28f77 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:22.924865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:22.926457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:22.940589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9675b-0368-4d39-8714-6d1ffbf331a6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.008288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.009110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.009391 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.009669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] PATH_INFO: `/v3/projects/71e9f04c57ac4e649d8dc96adb17caf8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.010895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e9f04c57ac4e649d8dc96adb17caf8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.012866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.030334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:23.072633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e9f04c57ac4e649d8dc96adb17caf8', 'request_id': 'req-9f246323-ab00-42ea-a44b-99389d615f99'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:23.080158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e9f04c57ac4e649d8dc96adb17caf8', 'request_id': 'req-9f246323-ab00-42ea-a44b-99389d615f99'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:23.088764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e9f04c57ac4e649d8dc96adb17caf8', 'request_id': 'req-9f246323-ab00-42ea-a44b-99389d615f99'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:23.100517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] The token cache is being invalidate because project 71e9f04c57ac4e649d8dc96adb17caf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:23.100826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:23.101482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9f246323-ab00-42ea-a44b-99389d615f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:23.104060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1230/2467] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:22 2022] DELETE /identity/v3/projects/71e9f04c57ac4e649d8dc96adb17caf8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:23.109918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.110202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.110565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.129999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:23.131219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:23.150080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:23.197102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1238/2468] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:23.206412 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.206679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.206962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.227650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:23.228911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:23.237920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1231/2469] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:23.242318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.252286 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9e306-514e-4289-a66e-48d7f8efe4da None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.256879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.257540 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.257822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.258097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.258875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:06:23.259120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.260771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.273035 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1239/2470] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:06:23.277573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.287887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab37b00-7034-4c4d-8d1c-7e62a08368a9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.292887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.293567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.293853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.294715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.295449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.297419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.325891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1232/2471] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:23.329989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.340646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18610d36-d7c7-4cec-bf45-aa3372f15f77 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.399670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.400578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.401040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.401519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.402840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.407133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.428817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1240/2472] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:23.433067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.443969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d214a-b8bc-4836-8319-d33dc0057db4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.448623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.449284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.449572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.449851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.451100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.452732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.463588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1233/2473] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:06:23.468302 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.478587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b3d9-292f-43f6-a1e0-b0fdc19b324f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.483403 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4043383-46d7-48a0-862e-68fe53696b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.484360 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043383-46d7-48a0-862e-68fe53696b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.484683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043383-46d7-48a0-862e-68fe53696b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.484964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043383-46d7-48a0-862e-68fe53696b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.485706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4043383-46d7-48a0-862e-68fe53696b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.487423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4043383-46d7-48a0-862e-68fe53696b7b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.499430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1241/2474] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:23.505857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.523287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009e2ef-d9b1-4cd6-b5f4-0cf5df19c33e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.529139 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.530038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.530312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.530580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] PATH_INFO: `/v3/projects/be1448a32792454a8d56693467c47023/users/05485a51da014f109c63bac42a3e7d5f/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.545723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1448a32792454a8d56693467c47023, user_id=05485a51da014f109c63bac42a3e7d5f, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.549961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.564833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1234/2475] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:23 2022] PUT /identity/v3/projects/be1448a32792454a8d56693467c47023/users/05485a51da014f109c63bac42a3e7d5f/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:23.575835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.576130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.576440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.605249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] MFA Rules not processed for user `05485a51da014f109c63bac42a3e7d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:23.606485 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:23.676528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1242/2476] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:06:23.681172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.743881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696929c0-28f6-48a7-ae76-c4d1680bdc93 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.748829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.749510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.749795 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.750073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.750778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.752742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.780813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1235/2477] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:23.784793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.801915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5212c-4fcc-493a-9c52-1da7951a8631 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.815714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:23.862937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.863603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.863883 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.864158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.864887 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.866786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.892500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1243/2478] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:23 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:23.896682 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.907807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e61da-af03-48c3-a096-8eb6189eb902 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.912657 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.913353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.913641 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.913921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.914667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.916307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.927816 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1236/2479] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:23.932137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.943216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25096c-f3b1-496b-8b81-f7b9685523e4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.948001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791506a-9b8c-4433-8bd8-5b967a8c31a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.948720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791506a-9b8c-4433-8bd8-5b967a8c31a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.949042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791506a-9b8c-4433-8bd8-5b967a8c31a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.949334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791506a-9b8c-4433-8bd8-5b967a8c31a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:23.950076 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791506a-9b8c-4433-8bd8-5b967a8c31a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:23.952287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791506a-9b8c-4433-8bd8-5b967a8c31a1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:23.964036 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1244/2480] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:23.969022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:23.986358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68541aa-f4b3-4c7d-9a00-06e0679fcdc1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:23.992499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:23.993435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:23.993711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:23.993983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] PATH_INFO: `/v3/projects/b41f07f91adf4d21a4319792d7c16a6e/users/47b605ad64ca4b789072ca5fb0554266/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.007397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:24.012303 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41f07f91adf4d21a4319792d7c16a6e, user_id=47b605ad64ca4b789072ca5fb0554266, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.017223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.032982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1237/2481] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:23 2022] PUT /identity/v3/projects/b41f07f91adf4d21a4319792d7c16a6e/users/47b605ad64ca4b789072ca5fb0554266/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:24.043974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.044455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.044814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.073417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] MFA Rules not processed for user `47b605ad64ca4b789072ca5fb0554266`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:24.074710 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:24.144833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1245/2482] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:24.149419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.212038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fb0d0-4390-4555-aaf8-e4af54d117ec admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.216791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.217455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.217736 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.218018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.218726 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.220639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.249092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1238/2483] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:24.253212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.264957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b439a0b-b4ab-4ba2-b9b2-740e90591335 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.323818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.324529 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.324816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.325093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.325813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.327734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.348345 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1246/2484] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:24 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:24.352581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.365567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75502e60-3d45-48a5-8c20-bbdfab2d6bf0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.370553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.371255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.371533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.371809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.373202 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.374834 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.386219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1239/2485] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:06:24.392304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.402963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16ab9b-42c3-4643-87c2-242dc30b8ac4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.408188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.409168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.409453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.409731 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] PATH_INFO: `/v3/projects/4a851e8da7f4435f9d7756e2cfc63124/users/1b1f1a20bb07476ea3adfe32ffe19308/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.425767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:24.430016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a851e8da7f4435f9d7756e2cfc63124, user_id=1b1f1a20bb07476ea3adfe32ffe19308, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.433107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.450343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1247/2486] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:24 2022] PUT /identity/v3/projects/4a851e8da7f4435f9d7756e2cfc63124/users/1b1f1a20bb07476ea3adfe32ffe19308/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:24.454488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.465818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e865-3556-43a4-9a50-e85d3656d2ac admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.527566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e69dad-1dbd-4300-88f7-3223d001a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.528780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e69dad-1dbd-4300-88f7-3223d001a14d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.529086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e69dad-1dbd-4300-88f7-3223d001a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.529367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e69dad-1dbd-4300-88f7-3223d001a14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.530117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e69dad-1dbd-4300-88f7-3223d001a14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.531754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e69dad-1dbd-4300-88f7-3223d001a14d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.542969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1240/2487] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:24.548210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.559142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26980d-61f1-46f1-9810-bce0a70e035e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.564188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.565139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.565419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.565700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] PATH_INFO: `/v3/projects/4a851e8da7f4435f9d7756e2cfc63124/users/1b1f1a20bb07476ea3adfe32ffe19308/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.567860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a851e8da7f4435f9d7756e2cfc63124, user_id=1b1f1a20bb07476ea3adfe32ffe19308, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.570584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.582249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1248/2488] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:24 2022] PUT /identity/v3/projects/4a851e8da7f4435f9d7756e2cfc63124/users/1b1f1a20bb07476ea3adfe32ffe19308/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:24.594035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.594409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.594754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.622715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] MFA Rules not processed for user `1b1f1a20bb07476ea3adfe32ffe19308`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:24.623962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:24.675091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:24.701481 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1241/2489] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:24.713024 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:24.714797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:24.776777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126781b-c414-4a5f-9843-64bd787b7d96 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:24.781042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:24.781692 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:24.781972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:24.782248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:24.843455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:24.844864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:24.855187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1249/2490] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:26.397376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:26.398854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:26.411973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeec76-3f35-47a9-9b9a-7345c4739bbb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:26.415619 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:26.416268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:26.416686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:26.416866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:26.477430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:26.478895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:26.489275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1242/2491] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:27.073419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:27.082513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606117-c512-41b9-b102-a3046d22be16 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:27.086264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530a86e-5a9b-46e3-99cb-d067b324debd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:27.086913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530a86e-5a9b-46e3-99cb-d067b324debd service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:27.087198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530a86e-5a9b-46e3-99cb-d067b324debd service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:27.087476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530a86e-5a9b-46e3-99cb-d067b324debd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:27.102724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530a86e-5a9b-46e3-99cb-d067b324debd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:27.104155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530a86e-5a9b-46e3-99cb-d067b324debd service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:27.115563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1250/2492] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:31.033749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:31.098452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99f5f-1222-4f3a-bffb-3f8034ba7c97 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:31.102976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:31.103843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:31.104120 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:31.104422 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] PATH_INFO: `/v3/users/05485a51da014f109c63bac42a3e7d5f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:31.105591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05485a51da014f109c63bac42a3e7d5f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:31.107509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:31.151915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05485a51da014f109c63bac42a3e7d5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.167735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05485a51da014f109c63bac42a3e7d5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.174483 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05485a51da014f109c63bac42a3e7d5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.181797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05485a51da014f109c63bac42a3e7d5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.182372 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05485a51da014f109c63bac42a3e7d5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.188176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1243/2493] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:31 2022] DELETE /identity/v3/users/05485a51da014f109c63bac42a3e7d5f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:31.192734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:31.193257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:31.193355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:31.210906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:31.212123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:31.270955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1251/2494] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:31.281677 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:31.283069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:31.288357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:31.343414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26529e5d-05a0-4ac8-9e80-eba5d748d637 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:31.347383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:31.348068 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:31.348353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:31.348664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:31.358250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:31.359972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:31.369581 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1244/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:31.653609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:31.655420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:31.668690 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555aaee-679e-4bb7-9845-bde60be213ad None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:31.673949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:31.674824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:31.675169 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:31.675454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] PATH_INFO: `/v3/projects/be1448a32792454a8d56693467c47023` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:31.676694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1448a32792454a8d56693467c47023)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:31.678798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:31.696705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:31.730688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1448a32792454a8d56693467c47023', 'request_id': 'req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.738094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1448a32792454a8d56693467c47023', 'request_id': 'req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.746642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1448a32792454a8d56693467c47023', 'request_id': 'req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.758501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] The token cache is being invalidate because project be1448a32792454a8d56693467c47023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:31.758916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.759589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.761225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1252/2496] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:31 2022] DELETE /identity/v3/projects/be1448a32792454a8d56693467c47023 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:31.766312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:31.776421 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753d3c8-47cd-4b53-8f78-ee92f9d553bf service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:31.843545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:31.844510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:31.844794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:31.845084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] PATH_INFO: `/v3/users/47b605ad64ca4b789072ca5fb0554266` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:31.846471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b605ad64ca4b789072ca5fb0554266)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:31.848441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:31.897724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b605ad64ca4b789072ca5fb0554266'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.910996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b605ad64ca4b789072ca5fb0554266'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.917904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b605ad64ca4b789072ca5fb0554266'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.927131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b605ad64ca4b789072ca5fb0554266'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.927913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b605ad64ca4b789072ca5fb0554266'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:31.934175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1245/2497] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:31 2022] DELETE /identity/v3/users/47b605ad64ca4b789072ca5fb0554266 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:32.201184 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:32.202621 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:32.216297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65c362-f9c2-4df8-bc2b-3bfa1dd43d62 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:32.273779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:32.274642 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:32.274924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:32.275198 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] PATH_INFO: `/v3/projects/b41f07f91adf4d21a4319792d7c16a6e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:32.276444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41f07f91adf4d21a4319792d7c16a6e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:32.278809 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:32.327173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41f07f91adf4d21a4319792d7c16a6e', 'request_id': 'req-fe775d77-41f5-4fe0-bb3c-1b469ba20247'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.333867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41f07f91adf4d21a4319792d7c16a6e', 'request_id': 'req-fe775d77-41f5-4fe0-bb3c-1b469ba20247'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.343368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41f07f91adf4d21a4319792d7c16a6e', 'request_id': 'req-fe775d77-41f5-4fe0-bb3c-1b469ba20247'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.354727 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] The token cache is being invalidate because project b41f07f91adf4d21a4319792d7c16a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:32.355037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.355666 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.357810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1253/2498] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:32 2022] DELETE /identity/v3/projects/b41f07f91adf4d21a4319792d7c16a6e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:32.361981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:32.363423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:32.376304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d3f6f-2fe7-4b8a-bfb1-8fed8787fb23 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:32.395932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:32.434420 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:32.435281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:32.435559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:32.435829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] PATH_INFO: `/v3/users/1b1f1a20bb07476ea3adfe32ffe19308` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:32.437022 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1f1a20bb07476ea3adfe32ffe19308)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:32.438951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:32.486550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1f1a20bb07476ea3adfe32ffe19308'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.499114 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1f1a20bb07476ea3adfe32ffe19308'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.506293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1f1a20bb07476ea3adfe32ffe19308'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.514500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1f1a20bb07476ea3adfe32ffe19308'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.515142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1f1a20bb07476ea3adfe32ffe19308'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.521129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1246/2499] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:32 2022] DELETE /identity/v3/users/1b1f1a20bb07476ea3adfe32ffe19308 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:32.538524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:32.540120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:32.580822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:32.628968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe775d77-41f5-4fe0-bb3c-1b469ba20247 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:32.724420 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:32.725113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:32.725512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:32.725674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:32.728588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:32.801734 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:32.803381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb94319-c1f8-497f-9f8b-1a38eb55b627 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:32.815655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1254/2500] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:32.838005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:32.841850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:32.919928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c752e0-ba39-4514-ae70-1c108fbe1855 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:32.924868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:32.925773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:32.926071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:32.926369 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] PATH_INFO: `/v3/projects/4a851e8da7f4435f9d7756e2cfc63124` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:32.927617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a851e8da7f4435f9d7756e2cfc63124)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:32.929686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:32.980029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:32.983926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a851e8da7f4435f9d7756e2cfc63124', 'request_id': 'req-3b9ee316-1972-4930-91f1-24a9e04f0f74'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:32.996500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a851e8da7f4435f9d7756e2cfc63124', 'request_id': 'req-3b9ee316-1972-4930-91f1-24a9e04f0f74'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:33.010010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a851e8da7f4435f9d7756e2cfc63124', 'request_id': 'req-3b9ee316-1972-4930-91f1-24a9e04f0f74'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:33.024381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] The token cache is being invalidate because project 4a851e8da7f4435f9d7756e2cfc63124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:33.024918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:33.025574 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3b9ee316-1972-4930-91f1-24a9e04f0f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:33.027138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1247/2501] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:32 2022] DELETE /identity/v3/projects/4a851e8da7f4435f9d7756e2cfc63124 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:33.032066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.032359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.032684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.051153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:33.052497 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:33.130764 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1255/2502] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:33.139337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.139629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.139912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.169947 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:33.171320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:33.180017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1248/2503] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:33.184465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.194748 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01db0-26dc-4b24-b7ba-6aa4412e79ae None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.199668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.200466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.200894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.201229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.202022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:06:33.202392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.204110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.206801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:33.216935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1256/2504] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:06:33.222147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.232152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ebb0d-dffc-43f2-a056-d15ae55ec2b4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.237127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.237788 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.238074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.238352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.239050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.241002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.268434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1249/2505] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:33.272770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.282922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55160f7f-1ccb-45b2-88f7-0fa48464b9e4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.339738 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.340329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.340680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.340968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.341674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.343608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.364609 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1257/2506] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:33.368430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.379639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccccbc8-d646-4599-8bae-0543ae8f4234 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.384229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.385004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.385282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.385561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.386326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.387918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.398820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1250/2507] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:33.402926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.413581 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a1841-4610-4b7c-b0df-290bf663954d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.420657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88ef0-4172-4ad0-a96b-438152d1d9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.421342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88ef0-4172-4ad0-a96b-438152d1d9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.421631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88ef0-4172-4ad0-a96b-438152d1d9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.421914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88ef0-4172-4ad0-a96b-438152d1d9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.422651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b88ef0-4172-4ad0-a96b-438152d1d9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.424314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b88ef0-4172-4ad0-a96b-438152d1d9ad admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.435934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1258/2508] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:33.440248 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.450609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a8b1-eb02-40ea-b39a-b0151c64307b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.455268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.456155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.456466 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.456748 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] PATH_INFO: `/v3/projects/efa32e84c77b4af1b31d4699a0a6f468/users/f1fe7725860c4ccea018a7e3e8243948/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.471566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa32e84c77b4af1b31d4699a0a6f468, user_id=f1fe7725860c4ccea018a7e3e8243948, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.474337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.488766 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1251/2509] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:33 2022] PUT /identity/v3/projects/efa32e84c77b4af1b31d4699a0a6f468/users/f1fe7725860c4ccea018a7e3e8243948/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:33.500478 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.500773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.501057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.527213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] MFA Rules not processed for user `f1fe7725860c4ccea018a7e3e8243948`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:33.529012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:33.595868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1259/2510] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:33.600232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.662562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3e18-a42d-421f-a6a0-0a546b773094 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.667283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.667946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.668229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.668551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.669872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.671772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.696972 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1252/2511] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:33.701000 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.711588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84533331-549c-42e2-96f4-07834a2fad0e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.768598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.769278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.769563 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.769838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.770544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.772482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.782077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:33.796040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1260/2512] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:33.799846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.800919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.803314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:33.811786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8119bf9-6284-4771-9d74-e93cbafe498c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.817745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.818431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.818708 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.818983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.819751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.821389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.835130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1253/2513] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:33.836530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.847258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb93a6-ffe3-4e9f-ba46-5545e9fc3a6d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.852512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.853176 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.853484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.853775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.854495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.856106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.867825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1254/2514] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:06:33.872099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:33.882403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf283a-1298-4b46-8383-1a82964b824e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.887213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959986-fe3f-4f1d-9f78-61c2c83fff12 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:33.888061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8da1f-8956-43df-ad2b-6c04a8c482c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.889081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8da1f-8956-43df-ad2b-6c04a8c482c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.889378 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8da1f-8956-43df-ad2b-6c04a8c482c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.889660 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8da1f-8956-43df-ad2b-6c04a8c482c1 admin admin] PATH_INFO: `/v3/projects/e60fc6fc7f6e4b5abf2109bfb02cb042/users/e56a646b00be4a88b1c9f09e1ff2b60e/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.894337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:33.895003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.895288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.895567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.897518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:33.907740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8da1f-8956-43df-ad2b-6c04a8c482c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60fc6fc7f6e4b5abf2109bfb02cb042, user_id=e56a646b00be4a88b1c9f09e1ff2b60e, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.910482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8da1f-8956-43df-ad2b-6c04a8c482c1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.925517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1255/2515] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:33 2022] PUT /identity/v3/projects/e60fc6fc7f6e4b5abf2109bfb02cb042/users/e56a646b00be4a88b1c9f09e1ff2b60e/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:33.935954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:33.936242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:33.936557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:33.959496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:33.967078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:33.968776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:33.971473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] MFA Rules not processed for user `e56a646b00be4a88b1c9f09e1ff2b60e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:33.974432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:33.979930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1261/2516] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:34.047169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1256/2517] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:34.051747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.063267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9b26c-5c43-48e9-b2de-7512cf4cd67c service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.128926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.129595 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.129880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.130157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.130844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.132786 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.172006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1262/2518] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:34 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:34.172625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.185239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7f67d-5aab-48c5-b455-84d62312d02d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.274324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.275009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.275293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.275573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.276274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.278335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.306112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1257/2519] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:34 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:34.310844 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.332564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11dc56-96a3-4260-8269-4be414bda4e7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.340538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.341519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.342222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.342751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.344279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.346756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.358918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1263/2520] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:34.363105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.373690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a20849-5d47-4319-95c4-c04891dd5b8e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.379679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.380614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.380901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.381260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] PATH_INFO: `/v3/projects/d945c1de79544e939d9aa4377c9349f5/users/eb10417f143142f6bcf8ebe0ca51add4/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.397590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d945c1de79544e939d9aa4377c9349f5, user_id=eb10417f143142f6bcf8ebe0ca51add4, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.403942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.421596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1258/2521] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:34 2022] PUT /identity/v3/projects/d945c1de79544e939d9aa4377c9349f5/users/eb10417f143142f6bcf8ebe0ca51add4/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:34.425843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.432326 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:34.437196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc391f-323a-4d4d-8cd1-5e1db0e061c6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.513574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93f253-0377-4c85-bc4c-9b5cc5675852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.514240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93f253-0377-4c85-bc4c-9b5cc5675852 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.514521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93f253-0377-4c85-bc4c-9b5cc5675852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.514790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93f253-0377-4c85-bc4c-9b5cc5675852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.515514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93f253-0377-4c85-bc4c-9b5cc5675852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.517156 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93f253-0377-4c85-bc4c-9b5cc5675852 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.528004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1264/2522] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:34.532236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.543093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499c1d-ccb0-4527-b2ac-389fc9ef4961 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.547835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.548762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.549044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.549324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] PATH_INFO: `/v3/projects/d945c1de79544e939d9aa4377c9349f5/users/eb10417f143142f6bcf8ebe0ca51add4/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.551353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d945c1de79544e939d9aa4377c9349f5, user_id=eb10417f143142f6bcf8ebe0ca51add4, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.554063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.567286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1259/2523] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:34 2022] PUT /identity/v3/projects/d945c1de79544e939d9aa4377c9349f5/users/eb10417f143142f6bcf8ebe0ca51add4/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:34.578944 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.579290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.579611 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.611572 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] MFA Rules not processed for user `eb10417f143142f6bcf8ebe0ca51add4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:34.613244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:34.695181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1265/2524] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:34.707029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:34.708933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:34.779847 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ef31-ef04-4642-ba27-1316cd3aeecd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:34.784219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:34.784931 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:34.785217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:34.785495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:34.790080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:34.852617 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:34.854072 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:34.864066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1260/2525] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:06:36.399871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:36.410760 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18ee07-e656-4c3a-9440-b7c595aa8cec None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:36.414428 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:36.415092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:36.415377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:36.415663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:36.478693 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:36.480147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:36.490551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1266/2526] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:37.025620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:37.034172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3d80-2840-4cc4-9e6a-3272532f813d service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:37.037849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e2420-fcac-4291-a7f1-714a8f82ac2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:37.038484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2420-fcac-4291-a7f1-714a8f82ac2b service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:37.039129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2420-fcac-4291-a7f1-714a8f82ac2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:37.039129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e2420-fcac-4291-a7f1-714a8f82ac2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:37.048548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e2420-fcac-4291-a7f1-714a8f82ac2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:37.049974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e2420-fcac-4291-a7f1-714a8f82ac2b service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:37.060247 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1261/2527] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:40.885677 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:40.929142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:40.992099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70433-1bd2-4a09-8632-24c8a4b3f275 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:40.996983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:40.998390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:40.998676 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:40.998957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] PATH_INFO: `/v3/users/f1fe7725860c4ccea018a7e3e8243948` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:41.000226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fe7725860c4ccea018a7e3e8243948)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:41.002249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:41.062657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fe7725860c4ccea018a7e3e8243948'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.078996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fe7725860c4ccea018a7e3e8243948'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.088316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fe7725860c4ccea018a7e3e8243948'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.089157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fe7725860c4ccea018a7e3e8243948'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.089859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fe7725860c4ccea018a7e3e8243948'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.104648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1267/2528] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:40 2022] DELETE /identity/v3/users/f1fe7725860c4ccea018a7e3e8243948 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:41.109797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:41.110596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:41.110596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:41.134444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:41.135741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:41.221310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1262/2529] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:41.228082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:41.229592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:41.301132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc20cf0-f28f-4ac1-b217-48d1db9595d7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:41.304996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:41.305675 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:41.305956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:41.306224 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:41.317409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:41.318952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:41.331173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1268/2530] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:06:41.675700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:41.678231 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:41.701094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db05bcf-5396-4f20-9ecb-75274235254b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:41.709100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:41.710573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:41.711073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:41.712543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] PATH_INFO: `/v3/projects/efa32e84c77b4af1b31d4699a0a6f468` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:41.714967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa32e84c77b4af1b31d4699a0a6f468)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:41.719680 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:41.775031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa32e84c77b4af1b31d4699a0a6f468', 'request_id': 'req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.781996 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa32e84c77b4af1b31d4699a0a6f468', 'request_id': 'req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.790844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa32e84c77b4af1b31d4699a0a6f468', 'request_id': 'req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.802547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] The token cache is being invalidate because project efa32e84c77b4af1b31d4699a0a6f468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:41.802832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.803502 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.805100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1263/2531] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:41 2022] DELETE /identity/v3/projects/efa32e84c77b4af1b31d4699a0a6f468 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:41.809500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:41.819845 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352d68f-0f0a-477e-b984-f99b5d770ab0 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:41.879392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:41.907369 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:41.908754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:41.908754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:41.909050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] PATH_INFO: `/v3/users/e56a646b00be4a88b1c9f09e1ff2b60e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:41.910296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56a646b00be4a88b1c9f09e1ff2b60e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:41.912445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:41.977525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56a646b00be4a88b1c9f09e1ff2b60e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:41.994544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56a646b00be4a88b1c9f09e1ff2b60e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.003974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56a646b00be4a88b1c9f09e1ff2b60e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.004956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56a646b00be4a88b1c9f09e1ff2b60e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.005658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56a646b00be4a88b1c9f09e1ff2b60e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.021624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1269/2532] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:41 2022] DELETE /identity/v3/users/e56a646b00be4a88b1c9f09e1ff2b60e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:42.380517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:42.383008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:42.407118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc900aca-a9d8-4cf9-b76f-fd312ed55db4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:42.438567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:42.537918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:42.539590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:42.540183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:42.540825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] PATH_INFO: `/v3/projects/e60fc6fc7f6e4b5abf2109bfb02cb042` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:42.543233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60fc6fc7f6e4b5abf2109bfb02cb042)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:42.549204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:42.644086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60fc6fc7f6e4b5abf2109bfb02cb042', 'request_id': 'req-6e30bf40-2384-4e62-896f-3c1d415f6909'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.651656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60fc6fc7f6e4b5abf2109bfb02cb042', 'request_id': 'req-6e30bf40-2384-4e62-896f-3c1d415f6909'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.660865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60fc6fc7f6e4b5abf2109bfb02cb042', 'request_id': 'req-6e30bf40-2384-4e62-896f-3c1d415f6909'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.674344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] The token cache is being invalidate because project e60fc6fc7f6e4b5abf2109bfb02cb042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:42.674634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.675355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.677137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1264/2533] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:42 2022] DELETE /identity/v3/projects/e60fc6fc7f6e4b5abf2109bfb02cb042 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:42.681081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:42.682739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:42.697368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fbefc-2c2d-4306-b66c-6769daa42a68 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:42.773186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:42.774105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:42.774388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:42.774665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] PATH_INFO: `/v3/users/eb10417f143142f6bcf8ebe0ca51add4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:42.775961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb10417f143142f6bcf8ebe0ca51add4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:42.777984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:42.835736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb10417f143142f6bcf8ebe0ca51add4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.849683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb10417f143142f6bcf8ebe0ca51add4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.852297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:42.858790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb10417f143142f6bcf8ebe0ca51add4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.859854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb10417f143142f6bcf8ebe0ca51add4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.861477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d3572d6-4ca4-4d7b-a0a8-1e448da60ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb10417f143142f6bcf8ebe0ca51add4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:42.875799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1270/2534] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:06:42 2022] DELETE /identity/v3/users/eb10417f143142f6bcf8ebe0ca51add4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:06:43.165027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:43.170443 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:43.189100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30bf40-2384-4e62-896f-3c1d415f6909 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:43.256034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:43.256944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.257228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.257506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] PATH_INFO: `/v3/projects/d945c1de79544e939d9aa4377c9349f5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.258716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d945c1de79544e939d9aa4377c9349f5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:43.260655 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:43.306381 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:43.307662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd945c1de79544e939d9aa4377c9349f5', 'request_id': 'req-734a0e61-cb28-46c9-b6ae-69158fcfc21b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:43.319668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd945c1de79544e939d9aa4377c9349f5', 'request_id': 'req-734a0e61-cb28-46c9-b6ae-69158fcfc21b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:43.330245 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd945c1de79544e939d9aa4377c9349f5', 'request_id': 'req-734a0e61-cb28-46c9-b6ae-69158fcfc21b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:43.342380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] The token cache is being invalidate because project d945c1de79544e939d9aa4377c9349f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:06:43.342656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:43.343341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-734a0e61-cb28-46c9-b6ae-69158fcfc21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:06:43.345004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1265/2535] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:06:43 2022] DELETE /identity/v3/projects/d945c1de79544e939d9aa4377c9349f5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:43.351704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.352201 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.352658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.380175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:43.381682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:43.454820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1271/2536] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:43.521993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.522672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.522672 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.543568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:43.544902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:43.553193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1266/2537] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:06:43.558610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:43.577792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a99d6-5a1c-41e0-ac0f-fa3564ccbb77 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:43.585762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:43.586960 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.587457 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.587921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.589327 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:06:43.589884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:43.592761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:43.615694 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1272/2538] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:06:43.620183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:43.631074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bde42c-06cc-47bf-b7bb-4ebcbdf6fcd4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:43.636279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:43.637013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.637286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.637817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.639129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:43.641156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:43.690195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1267/2539] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:06:43 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:06:43.699298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:43.718187 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98253bfe-78c3-485b-9710-ddfe78b482bb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:43.832662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:43.833696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.834217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.834395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.835042 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:43.837135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:43.873881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1273/2540] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:06:43 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:06:43.881205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:43.904649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45763bfe-4be1-4aad-86ca-2673707580a6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:43.912751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:43.913901 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.914367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.914833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.916152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:43.919709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:43.944439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1268/2541] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:43.946097 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:43.957350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216c790-b249-4812-854a-06fd4a87d0b4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:43.962674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc294c7b-75dc-4dc0-92d0-3090cab56135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:43.963405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc294c7b-75dc-4dc0-92d0-3090cab56135 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:43.963691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc294c7b-75dc-4dc0-92d0-3090cab56135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:43.963965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc294c7b-75dc-4dc0-92d0-3090cab56135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:43.964750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc294c7b-75dc-4dc0-92d0-3090cab56135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:43.966414 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc294c7b-75dc-4dc0-92d0-3090cab56135 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:43.979141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1274/2542] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:06:43.991981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:44.007647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b897-ed06-4f37-bef8-98e499464c37 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:44.015649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:44.017149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:44.017626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:44.018259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] PATH_INFO: `/v3/projects/a63341c013394ac29ff8405711b0d6d9/users/a47c19f7d90b4b9a83e5ddf8627acf2f/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:44.043994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63341c013394ac29ff8405711b0d6d9, user_id=a47c19f7d90b4b9a83e5ddf8627acf2f, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:44.048328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:44.071646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1269/2543] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:06:43 2022] PUT /identity/v3/projects/a63341c013394ac29ff8405711b0d6d9/users/a47c19f7d90b4b9a83e5ddf8627acf2f/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:06:44.084974 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad1055-dc09-449b-9b5d-a9eaf5ab7d62 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:44.085265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad1055-dc09-449b-9b5d-a9eaf5ab7d62 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:44.085545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad1055-dc09-449b-9b5d-a9eaf5ab7d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:44.123784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-07ad1055-dc09-449b-9b5d-a9eaf5ab7d62 None None] MFA Rules not processed for user `a47c19f7d90b4b9a83e5ddf8627acf2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:44.125058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-07ad1055-dc09-449b-9b5d-a9eaf5ab7d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:44.194118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:44.231093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1275/2544] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:44.245523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:44.248134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:44.309654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2693d-c193-4ef5-8bd6-38702c20df0b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:44.313578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:44.314232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:44.314533 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:44.314807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:44.323702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:44.325165 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:44.336185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1270/2545] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:45.729338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:45.729872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:45.730002 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:45.749718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] MFA Rules not processed for user `a47c19f7d90b4b9a83e5ddf8627acf2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:45.750955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:45.758011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1276/2546] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:46.244565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:46.246163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:46.348716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec1fd3-33ca-43e9-a1b1-deb33265d5b6 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:46.355035 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:46.355706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:46.355988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:46.356264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:46.358832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:46.371835 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:46.373548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:46.384801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1271/2547] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:48.581455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:48.583138 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:48.657287 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e91806-60bd-4d49-8db4-445c248a2d23 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:48.661447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:48.662200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:48.662549 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:48.662893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:48.665189 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:48.677528 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:48.679111 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:48.689924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1277/2548] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:49.358656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:49.360368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:49.369056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:06:49.463895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e484c-a726-4183-b252-fb5afbdd0478 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:49.470805 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:49.471578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:49.471950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:49.472538 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:49.475941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:49.490541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:49.492325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e45688-1995-485f-9fbc-a84cf6b3a4ab service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:49.505169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1272/2549] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:06:54.150226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:54.168251 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc579013-26fc-4d34-8025-71b941251be1 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:54.175868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:54.176664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:54.177014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:54.177366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:54.179757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:54.193950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:54.195698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:54.209124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1278/2550] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:06:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:06:54.539498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5fdc5-26a4-4797-8489-42f9f650f441 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:54.539869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5fdc5-26a4-4797-8489-42f9f650f441 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:54.540042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5fdc5-26a4-4797-8489-42f9f650f441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:54.564020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-79f5fdc5-26a4-4797-8489-42f9f650f441 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:06:54.565122 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79f5fdc5-26a4-4797-8489-42f9f650f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:54.639138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1273/2551] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:06:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:06:54.652925 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:06:54.654854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:06:54.726474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e441-0154-4bef-ace5-efc7f326e3f8 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:06:54.732352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:06:54.733081 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:06:54.733364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:06:54.733799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:06:54.744041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:06:54.745591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:06:54.754131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1279/2552] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:07:04.273251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24838830-c183-4054-a243-ce8ef46ff40c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:04.274365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24838830-c183-4054-a243-ce8ef46ff40c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:04.274717 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24838830-c183-4054-a243-ce8ef46ff40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:04.310286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-24838830-c183-4054-a243-ce8ef46ff40c None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:04.313033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-24838830-c183-4054-a243-ce8ef46ff40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:04.323153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1274/2553] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:07:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:04.338107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:04.354705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27039534-42f3-4856-ad74-f351048ec537 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:04.361372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:04.362292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:04.362896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:04.363180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:04.373927 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:04.375652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:04.386180 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1280/2554] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:07:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:07:05.131825 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4172c-e9fc-4f0e-8a46-75fe1540cecf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:05.132118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4172c-e9fc-4f0e-8a46-75fe1540cecf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:05.132445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4172c-e9fc-4f0e-8a46-75fe1540cecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:05.156892 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-17d4172c-e9fc-4f0e-8a46-75fe1540cecf None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:05.158545 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-17d4172c-e9fc-4f0e-8a46-75fe1540cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:05.168552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1275/2555] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jul 15 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:05.181350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:05.194633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd873b-1d24-44ed-a8d4-3fc058cd4147 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:05.200768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:05.204121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:05.204121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:05.204121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:05.215548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:05.217380 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:05.226633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1281/2556] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:07:05.456722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3c2c-7e3f-4ca7-aa7f-d43fb16b2c66 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:05.457258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3c2c-7e3f-4ca7-aa7f-d43fb16b2c66 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:05.457258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3c2c-7e3f-4ca7-aa7f-d43fb16b2c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:05.478678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d2df3c2c-7e3f-4ca7-aa7f-d43fb16b2c66 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:05.479910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2df3c2c-7e3f-4ca7-aa7f-d43fb16b2c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:05.488308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1276/2557] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:05.502011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:05.513966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e376d-b984-4844-8602-74a283c5a856 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:05.520431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:05.521681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:05.521962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:05.522236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:05.532275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:05.533907 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:05.543031 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1282/2558] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:07:05.983521 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:05.983812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:05.984093 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:06.009691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:06.010935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:06.019604 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1277/2559] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jul 15 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:06.032079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:06.043363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350384ca-014c-4c64-b7da-04d3e311c7b4 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:06.048640 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3a664-507e-4876-9c02-7e4c3c90f92e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:06.049309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3a664-507e-4876-9c02-7e4c3c90f92e service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:06.049645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3a664-507e-4876-9c02-7e4c3c90f92e service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:06.050129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3a664-507e-4876-9c02-7e4c3c90f92e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:06.059841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3a664-507e-4876-9c02-7e4c3c90f92e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:06.062752 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3a664-507e-4876-9c02-7e4c3c90f92e service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:06.068233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:06.073379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1283/2560] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:07:08.202653 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:08.270870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed99eb3-4971-4b84-8e62-6dc4e262f159 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:08.275394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:08.276276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:08.276586 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:08.276865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] PATH_INFO: `/v3/users/a47c19f7d90b4b9a83e5ddf8627acf2f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:08.278084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47c19f7d90b4b9a83e5ddf8627acf2f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:08.280033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:08.414738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:08.452706 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47c19f7d90b4b9a83e5ddf8627acf2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:08.467893 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47c19f7d90b4b9a83e5ddf8627acf2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:08.477499 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47c19f7d90b4b9a83e5ddf8627acf2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:08.490009 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47c19f7d90b4b9a83e5ddf8627acf2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:08.490692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47c19f7d90b4b9a83e5ddf8627acf2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:08.497827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1278/2561] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:07:08 2022] DELETE /identity/v3/users/a47c19f7d90b4b9a83e5ddf8627acf2f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:08.503646 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:08.504222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:08.504352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:08.522455 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:08.523689 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:08.587427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1284/2562] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:07:08.599771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:08.601324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:08.660279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f674c1-9b44-45f3-9d85-33b6b567b121 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:08.664049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2e133-4215-422c-9c3f-3df57d89b2fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:08.664727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2e133-4215-422c-9c3f-3df57d89b2fc service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:08.665034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2e133-4215-422c-9c3f-3df57d89b2fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:08.665300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2e133-4215-422c-9c3f-3df57d89b2fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:08.675771 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2e133-4215-422c-9c3f-3df57d89b2fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:08.677442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2e133-4215-422c-9c3f-3df57d89b2fc service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:08.688446 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1279/2563] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:08.864862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:08.866282 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:08.887060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03300a8c-589f-4b96-a5e2-b464b83adb5e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:08.892616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:08.893553 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:08.893880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:08.894102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] PATH_INFO: `/v3/projects/a63341c013394ac29ff8405711b0d6d9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:08.896319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63341c013394ac29ff8405711b0d6d9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:08.899843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:08.985110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63341c013394ac29ff8405711b0d6d9', 'request_id': 'req-6198d3f9-0770-4937-acfb-daa16e22c8a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:08.991891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63341c013394ac29ff8405711b0d6d9', 'request_id': 'req-6198d3f9-0770-4937-acfb-daa16e22c8a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:09.000594 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63341c013394ac29ff8405711b0d6d9', 'request_id': 'req-6198d3f9-0770-4937-acfb-daa16e22c8a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:09.021025 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] The token cache is being invalidate because project a63341c013394ac29ff8405711b0d6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:07:09.021932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:09.022502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6198d3f9-0770-4937-acfb-daa16e22c8a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:09.030363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1285/2564] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:07:08 2022] DELETE /identity/v3/projects/a63341c013394ac29ff8405711b0d6d9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:09.032312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.032724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.033094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.054131 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:09.055356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:09.126575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1280/2565] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:09.134044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102196-53f9-400f-b769-0068382db63d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.134336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102196-53f9-400f-b769-0068382db63d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.134715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3102196-53f9-400f-b769-0068382db63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.157069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f3102196-53f9-400f-b769-0068382db63d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:09.158313 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f3102196-53f9-400f-b769-0068382db63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:09.168283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1286/2566] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:09.172894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.183222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a84e3b-c7d5-461b-9721-392ff845bedf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.189038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.189849 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.190005 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.190292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.191025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:07:09.191362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.193004 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.206084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1281/2567] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:07:09.210684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3102196-53f9-400f-b769-0068382db63d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.221046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3102196-53f9-400f-b769-0068382db63d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.226113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.226780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.227092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.227373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.228077 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.230021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.256249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1287/2568] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:07:09 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:07:09.260343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.270824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41ab7-7e52-4474-a01f-72652b890402 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.282864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:09.332786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.333453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.333738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.334015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.334723 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.336651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.355566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1282/2569] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:07:09 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:07:09.360116 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.370888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e8024-d623-4056-b9fd-7bdbb54e45a6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.376057 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.376767 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.377049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.377328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.378063 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.379688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.392511 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1288/2570] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:07:09.396406 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.407285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a67d8-64ac-459f-a386-bc13c5cc623e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.412310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ded69-a8a1-48dc-8dfe-dce536db476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.413070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ded69-a8a1-48dc-8dfe-dce536db476d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.413562 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ded69-a8a1-48dc-8dfe-dce536db476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.414059 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ded69-a8a1-48dc-8dfe-dce536db476d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.415488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ded69-a8a1-48dc-8dfe-dce536db476d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.417115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ded69-a8a1-48dc-8dfe-dce536db476d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.430927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1283/2571] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:07:09.435467 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.447274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba23e-d801-46a1-9396-ba7ca4ba0405 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.453392 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.454356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.454643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.454922 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] PATH_INFO: `/v3/projects/f4031bb6048646cb9b9bab738abf7203/users/067e49e0e6f8451a885481641bd9df83/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.471398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4031bb6048646cb9b9bab738abf7203, user_id=067e49e0e6f8451a885481641bd9df83, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.474105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.489610 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1289/2572] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:07:09 2022] PUT /identity/v3/projects/f4031bb6048646cb9b9bab738abf7203/users/067e49e0e6f8451a885481641bd9df83/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:07:09.500559 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.500853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.501265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.529228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] MFA Rules not processed for user `067e49e0e6f8451a885481641bd9df83`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:09.529827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:09.599506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1284/2573] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:07:09.613496 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:09.615305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:09.643440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:09.696875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315788c2-bb4f-4595-8361-5455b035dbb0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:09.701041 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:09.701929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:09.702407 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:09.702746 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:09.713311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:09.714867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:09.725240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1290/2574] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:10.173511 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:10.178036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:10.252295 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728e07-9010-4936-a740-25bf138b6e6e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:10.258402 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:10.260734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:10.261506 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:10.262253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:10.265752 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:10.347853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:10.383854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:10.385667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:10.398382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1285/2575] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:10.572739 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:10.574195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:10.592435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64a210-2c7c-4e31-b2b8-870c000ef07e service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:10.596473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:10.597153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:10.597433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:10.597713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:10.599525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:10.610439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:10.611918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:10.621080 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1291/2576] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:13.922821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:13.924333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:13.981641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:13.985329 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:14.005174 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ccf9-9305-4538-bf5b-f468b464c4ef service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.015831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.017451 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.017982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f34215-be7f-4de1-ae64-2c887fdfce2b service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.019230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.019230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] PATH_INFO: `/v3/users/96d4c389c4da4d2188e8a32e5ed1be7d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.020814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d4c389c4da4d2188e8a32e5ed1be7d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.022686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.023332 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.023633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.023890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.024443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.026494 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:14.040915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.042523 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.053797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1286/2577] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:14.081963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d4c389c4da4d2188e8a32e5ed1be7d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.095154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d4c389c4da4d2188e8a32e5ed1be7d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.102573 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d4c389c4da4d2188e8a32e5ed1be7d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.103214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d4c389c4da4d2188e8a32e5ed1be7d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.103830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-77d53361-aeeb-42c1-b0c0-eac66a7ce88a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d4c389c4da4d2188e8a32e5ed1be7d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.116882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1292/2578] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:07:13 2022] DELETE /identity/v3/users/96d4c389c4da4d2188e8a32e5ed1be7d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:14.295124 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.306645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbda97-3ca6-4371-9bef-da4dfc8619e0 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.382921 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.383896 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.384269 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.384638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] PATH_INFO: `/v3/projects/c8778d5da3de4c4dbae5d24842b359bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.385872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8778d5da3de4c4dbae5d24842b359bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.387780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.442935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8778d5da3de4c4dbae5d24842b359bc', 'request_id': 'req-918e0062-e046-493b-9e6d-779166f97237'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.449213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8778d5da3de4c4dbae5d24842b359bc', 'request_id': 'req-918e0062-e046-493b-9e6d-779166f97237'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.457697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8778d5da3de4c4dbae5d24842b359bc', 'request_id': 'req-918e0062-e046-493b-9e6d-779166f97237'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.463186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:14.470939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] The token cache is being invalidate because project c8778d5da3de4c4dbae5d24842b359bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:07:14.471590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.472867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-918e0062-e046-493b-9e6d-779166f97237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:14.474590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1287/2579] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:07:14 2022] DELETE /identity/v3/projects/c8778d5da3de4c4dbae5d24842b359bc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:14.479905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.480303 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.480769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.499463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:14.500698 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:14.564256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1293/2580] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:14.573264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.573264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.573615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.593335 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:14.594560 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:14.602437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1288/2581] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:14.606840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.617163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b070b2-a5a1-47c8-b668-5cec31145a8c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.621751 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.622527 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.622810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.623586 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.624346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:07:14.624681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.626288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.638185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1294/2582] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:07:14.643237 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.652553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d347e-748b-451b-8bd5-b7eb87c6e1be None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.657388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.658046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.658328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.658603 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.659350 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.661286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.687189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1289/2583] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:07:14 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:07:14.691465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.702432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3fbed-3a80-43fc-b4f0-b771fa34d19b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.712056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:14.772558 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.773245 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.773536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.773814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.774519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.779083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.801033 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1295/2584] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:07:14 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:07:14.807550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.816920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19f3dc-9266-4987-93f4-0731723a62b6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.821624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.822307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.822602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.822874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.823624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.825284 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.838980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1290/2585] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:07:14.843108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.853448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872c840-b217-4993-9c33-6652a1146572 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.861479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9019e-da7d-4c83-9a74-d20ad631e8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.862816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9019e-da7d-4c83-9a74-d20ad631e8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.863109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9019e-da7d-4c83-9a74-d20ad631e8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.863874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9019e-da7d-4c83-9a74-d20ad631e8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.864650 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9019e-da7d-4c83-9a74-d20ad631e8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.866311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9019e-da7d-4c83-9a74-d20ad631e8cf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.882109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1296/2586] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:07:14.886220 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:14.896549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22abe9-2c4b-4f8f-9b6b-d6a6461222cc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:14.904824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:14.905838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.906190 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.906590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] PATH_INFO: `/v3/projects/26daacf03c844ff8a7546dfe3b36d8bb/users/4a3a211465a0478994d011f89eaf972f/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.923643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26daacf03c844ff8a7546dfe3b36d8bb, user_id=4a3a211465a0478994d011f89eaf972f, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:14.926447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:14.951051 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1291/2587] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:07:14 2022] PUT /identity/v3/projects/26daacf03c844ff8a7546dfe3b36d8bb/users/4a3a211465a0478994d011f89eaf972f/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:14.956350 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15a25a-b2c7-4e8f-ab30-e5cacd98efc0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:14.956665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15a25a-b2c7-4e8f-ab30-e5cacd98efc0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:14.956953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15a25a-b2c7-4e8f-ab30-e5cacd98efc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:14.975388 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eb15a25a-b2c7-4e8f-ab30-e5cacd98efc0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:14.976664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eb15a25a-b2c7-4e8f-ab30-e5cacd98efc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:15.036653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1297/2588] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:15.047127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:15.049821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:15.146065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51ae34-e97d-4d62-8685-d68307a3d01a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:15.153252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:15.153552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:15.153868 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:15.154061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:15.156177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:15.156177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:15.157180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:15.176754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:15.178378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:15.179659 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:15.179869 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:15.193411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1292/2589] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:15.248813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1298/2590] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:07:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:15.277356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:15.278936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:15.361826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47c82-615e-4104-a033-6669d805f264 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:15.367034 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6ecbb-ce07-414e-b201-e3e863bfc290 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:15.368069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6ecbb-ce07-414e-b201-e3e863bfc290 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:15.368069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6ecbb-ce07-414e-b201-e3e863bfc290 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:15.368306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6ecbb-ce07-414e-b201-e3e863bfc290 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:15.377854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6ecbb-ce07-414e-b201-e3e863bfc290 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:15.380590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6ecbb-ce07-414e-b201-e3e863bfc290 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:15.394195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1293/2591] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:07:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 14:07:17.286299 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:17.287776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:17.320921 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcec96-8f38-4e53-b12a-8af95c0a2291 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:17.332983 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:17.346827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:17.347349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:17.347717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] PATH_INFO: `/v3/users/067e49e0e6f8451a885481641bd9df83` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:17.353445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=067e49e0e6f8451a885481641bd9df83)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:17.361084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:17.448315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:17.478055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067e49e0e6f8451a885481641bd9df83'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:17.496957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067e49e0e6f8451a885481641bd9df83'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:17.513385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067e49e0e6f8451a885481641bd9df83'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:17.515253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067e49e0e6f8451a885481641bd9df83'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:17.517513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067e49e0e6f8451a885481641bd9df83'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:17.543396 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1299/2592] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:07:17 2022] DELETE /identity/v3/users/067e49e0e6f8451a885481641bd9df83 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:17.548576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:17.549106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:17.549221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:17.616625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:17.620316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:17.692517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:17.815724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1294/2593] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:17.832346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:17.836211 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:17.941878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a4eae-6fb2-4f89-a38b-29bac6512c6f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:17.951801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf914-e163-4bce-bb6d-be47faa45066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:17.952994 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf914-e163-4bce-bb6d-be47faa45066 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:17.953889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf914-e163-4bce-bb6d-be47faa45066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:17.954387 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf914-e163-4bce-bb6d-be47faa45066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:17.974566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf914-e163-4bce-bb6d-be47faa45066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:17.977665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf914-e163-4bce-bb6d-be47faa45066 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:17.999234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1300/2594] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:18.924860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:18.926339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:18.954555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e011387-3a04-455e-8a9c-3e6ebc9dc72d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:18.968938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:18.970065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:18.970353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:18.970626 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] PATH_INFO: `/v3/projects/f4031bb6048646cb9b9bab738abf7203` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:18.974256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4031bb6048646cb9b9bab738abf7203)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:18.979049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:19.107141 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4031bb6048646cb9b9bab738abf7203', 'request_id': 'req-4ab647b5-605a-413a-88c4-dfa5ede3954d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:19.120973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4031bb6048646cb9b9bab738abf7203', 'request_id': 'req-4ab647b5-605a-413a-88c4-dfa5ede3954d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:19.151722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4031bb6048646cb9b9bab738abf7203', 'request_id': 'req-4ab647b5-605a-413a-88c4-dfa5ede3954d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:19.176542 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] The token cache is being invalidate because project f4031bb6048646cb9b9bab738abf7203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:07:19.176839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:19.177690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4ab647b5-605a-413a-88c4-dfa5ede3954d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:07:19.179585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1295/2595] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:07:18 2022] DELETE /identity/v3/projects/f4031bb6048646cb9b9bab738abf7203 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:07:19.185053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.185612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.185612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.218670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:19.223743 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:19.436958 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1301/2596] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:19.445512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.446074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.446519 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.484043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:19.487087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:19.504006 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1296/2597] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:19.504729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:19.519040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c72c5f-b2f8-49e3-bee5-cffc3738d108 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:19.528483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:19.530082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.530673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.531409 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.533273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:07:19.534005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:19.538466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:19.565951 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:19.580023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1302/2598] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:07:19.581191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9606fc1-195f-4d4d-a066-eb22719f6526 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:19.589856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:19.591371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.591929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.593382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.596670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:19.606596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:19.685223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1297/2599] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:07:19 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:07:19.691288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:19.703105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c529bd-388f-408f-8560-9a7017ebe97b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:19.800704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:19.868926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:19.869881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.870405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.870908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.872007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:19.874879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:19.924407 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:19.936815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1303/2600] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:07:19 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:07:19.938384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7073a8b-7eee-4dc5-9f7b-97f5c9f1a3d5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:19.945033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:19.946121 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.946625 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.947232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.948359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:19.950448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:19.969253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:19.982740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153859e6-02e8-45dd-b700-5b7bddf1d384 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:19.983484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1298/2601] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:07:19.989855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f5b58-3e6e-4525-99b3-2a77ef815484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:19.991137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f5b58-3e6e-4525-99b3-2a77ef815484 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:19.991669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f5b58-3e6e-4525-99b3-2a77ef815484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:19.992170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f5b58-3e6e-4525-99b3-2a77ef815484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:19.993188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f5b58-3e6e-4525-99b3-2a77ef815484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:19.995050 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f5b58-3e6e-4525-99b3-2a77ef815484 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:20.014157 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:20.025699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3d30f-567d-44f3-b698-4c13b60a7ee9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:20.026476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1304/2602] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:07:20.033331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:20.034954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:20.035549 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:20.036097 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] PATH_INFO: `/v3/projects/e608cc75a5ed4f05b462c84c29a31788/users/f72d7744f4c4493a8e2a5f87e9a89d5c/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:20.056794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e608cc75a5ed4f05b462c84c29a31788, user_id=f72d7744f4c4493a8e2a5f87e9a89d5c, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:20.060255 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:20.086995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1299/2603] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:07:20 2022] PUT /identity/v3/projects/e608cc75a5ed4f05b462c84c29a31788/users/f72d7744f4c4493a8e2a5f87e9a89d5c/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:20.092832 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07d6ce-6071-4a70-b83e-e4929b6dfea0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:20.093290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07d6ce-6071-4a70-b83e-e4929b6dfea0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:20.093686 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07d6ce-6071-4a70-b83e-e4929b6dfea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:20.123398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8e07d6ce-6071-4a70-b83e-e4929b6dfea0 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:20.124429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8e07d6ce-6071-4a70-b83e-e4929b6dfea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:20.213458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1305/2604] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:20.225037 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:20.227798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:20.339251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:20.371791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2f5a5-0ee3-4e95-ac95-f6681a93837b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:20.384316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:20.385156 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:20.385441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:20.385719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:20.409423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:20.411324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:20.434168 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1300/2605] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:24.054479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44453f-a24c-4868-bb83-e69da67a84a1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:24.055503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44453f-a24c-4868-bb83-e69da67a84a1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:24.056100 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44453f-a24c-4868-bb83-e69da67a84a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:24.114082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0c44453f-a24c-4868-bb83-e69da67a84a1 None None] MFA Rules not processed for user `4a3a211465a0478994d011f89eaf972f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:24.115848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c44453f-a24c-4868-bb83-e69da67a84a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:24.262806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1306/2606] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:24.284853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:24.287504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:24.464460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269350de-8ee3-4a16-b520-f1d544139ecf service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:24.471268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219a185-8c9a-4004-9978-2023f8df38ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:24.472102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219a185-8c9a-4004-9978-2023f8df38ae service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:24.472567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219a185-8c9a-4004-9978-2023f8df38ae service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:24.473065 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219a185-8c9a-4004-9978-2023f8df38ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:24.483417 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219a185-8c9a-4004-9978-2023f8df38ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:24.485547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219a185-8c9a-4004-9978-2023f8df38ae service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:24.496879 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1301/2607] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:26.973670 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:26.974205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:26.974306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:27.015554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] MFA Rules not processed for user `4a3a211465a0478994d011f89eaf972f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:27.016873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:27.025110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1307/2608] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:28.047080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0315f-913c-4779-8373-26c103f7269a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.050297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0315f-913c-4779-8373-26c103f7269a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.050297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0315f-913c-4779-8373-26c103f7269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.108374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-79b0315f-913c-4779-8373-26c103f7269a None None] MFA Rules not processed for user `f72d7744f4c4493a8e2a5f87e9a89d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:28.111889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79b0315f-913c-4779-8373-26c103f7269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:28.283515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1302/2609] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:28.288663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:28.307588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacddf60-0da4-4b20-991d-66aea364735f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:28.316481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:28.317592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.318018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.318018 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.318649 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:28.326136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:28.374465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1308/2610] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:07:28 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:07:28.375899 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0315f-913c-4779-8373-26c103f7269a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:28.391018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0315f-913c-4779-8373-26c103f7269a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:28.529319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:28.530019 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.530485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.530770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.531527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:28.533544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:28.543435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:28.563287 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1303/2611] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:07:28 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:07:28.567172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:28.580483 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0f390-b58d-461e-8082-60d3a421ac84 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:28.586660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:28.588102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.588652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.589238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.590356 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:28.592492 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:28.609336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1309/2612] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:07:28.614692 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:28.636280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab0e58-96dd-4578-b4f7-08df6047c73b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:28.644102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:28.645392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.645783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.646137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] PATH_INFO: `/v3/projects/45cc3efc2301469d897af308ce66fde6/users/be679ac36101449d8bad2fef00783737/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.672668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45cc3efc2301469d897af308ce66fde6, user_id=be679ac36101449d8bad2fef00783737, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:28.675994 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:28.704170 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1304/2613] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:07:28 2022] PUT /identity/v3/projects/45cc3efc2301469d897af308ce66fde6/users/be679ac36101449d8bad2fef00783737/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:28.705705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:28.714207 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:28.719493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c4d82-5a1b-4cb5-8452-31731d627900 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:28.800214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e32091-05fd-4f85-a619-4846dc0500bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:28.801075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e32091-05fd-4f85-a619-4846dc0500bf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.801889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e32091-05fd-4f85-a619-4846dc0500bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.802354 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e32091-05fd-4f85-a619-4846dc0500bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.803461 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e32091-05fd-4f85-a619-4846dc0500bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:28.805301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e32091-05fd-4f85-a619-4846dc0500bf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:28.820956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1310/2614] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:07:28.824928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:28.837325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541a053-0735-4c19-9375-afbca9c0c001 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:28.845150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:28.846085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:28.846479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:28.846764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] PATH_INFO: `/v3/projects/45cc3efc2301469d897af308ce66fde6/users/be679ac36101449d8bad2fef00783737/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:28.850782 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45cc3efc2301469d897af308ce66fde6, user_id=be679ac36101449d8bad2fef00783737, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:28.854370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:28.888321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1305/2615] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:07:28 2022] PUT /identity/v3/projects/45cc3efc2301469d897af308ce66fde6/users/be679ac36101449d8bad2fef00783737/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:07:33.425167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f57fc-6fa8-4968-bed3-441f093f3646 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:33.426060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f57fc-6fa8-4968-bed3-441f093f3646 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:33.426575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f57fc-6fa8-4968-bed3-441f093f3646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:33.469249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3e8f57fc-6fa8-4968-bed3-441f093f3646 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:33.470699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3e8f57fc-6fa8-4968-bed3-441f093f3646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:33.619923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1311/2616] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jul 15 14:07:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:33.632163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:33.634753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:33.802765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176843b-0b0b-4fe2-84b8-c74337852f5d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:33.812177 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:33.812910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:33.813196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:33.813472 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:33.828117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:33.829810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:33.852243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1306/2617] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:07:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:07:35.825342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5c153-758f-474a-ace4-6c3903a7e9a3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:35.826466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5c153-758f-474a-ace4-6c3903a7e9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:35.826466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5c153-758f-474a-ace4-6c3903a7e9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:35.861013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-66a5c153-758f-474a-ace4-6c3903a7e9a3 None None] MFA Rules not processed for user `be679ac36101449d8bad2fef00783737`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:35.864871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-66a5c153-758f-474a-ace4-6c3903a7e9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:35.970079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1312/2618] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:35.984246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:36.095843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cc0f3-76b3-4a54-885c-fb1abeafa094 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:36.101961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:36.103018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:36.103471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:36.103754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:36.152987 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:36.175442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:36.177194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:36.187830 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1307/2619] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:37.278291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:37.278777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:37.278777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:37.298108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] MFA Rules not processed for user `f72d7744f4c4493a8e2a5f87e9a89d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:07:37.299338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:37.307445 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1313/2620] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:07:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:07:42.760064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:07:42.772084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269aa73-67c0-46af-b3d5-58073b562b0a service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:07:42.777768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:07:42.778588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:07:42.779643 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:07:42.779919 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:07:42.781765 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:07:42.794307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:07:42.795944 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:07:42.806832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1308/2621] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:07:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:07:47.291212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:07:52.802296 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=15 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:09.954168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:09.956019 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:10.015759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ded94f-a860-42c5-b297-f5ee5e74a478 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:10.019801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:10.020916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:10.021265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:10.021592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:10.031562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:10.033060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:10.042975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1314/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:16.068594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:16.079214 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32e005-ade0-4050-9ed8-6bf5377539ac service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:16.083008 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:16.083713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:16.083942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:16.084217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:16.086036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:16.153281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:16.154930 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:16.165778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1309/2623] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:19.474023 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:19.475566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:19.553221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:19.555812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:19.584928 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc80ae1-0d49-41b3-9b19-adf92b9ab8d5 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:19.590665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:19.591667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:19.591950 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:19.592227 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] PATH_INFO: `/v3/users/520583af1f264f22a86722b0cb9ff405` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:19.593801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] RBAC: Authorizing `identity:delete_user(user_id=520583af1f264f22a86722b0cb9ff405)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:19.596129 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:19.604655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935cf19-d68c-49f2-8c3c-4ad0a15851f0 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:19.611902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:19.612990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:19.613379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:19.613674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:19.616532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:19.633040 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:19.635152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:19.649043 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1315/2624] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:19.672438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520583af1f264f22a86722b0cb9ff405'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:19.689055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520583af1f264f22a86722b0cb9ff405'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:19.698743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520583af1f264f22a86722b0cb9ff405'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:19.707887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520583af1f264f22a86722b0cb9ff405'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:19.709389 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2ef537cc-f880-47a3-9d12-a2992789085a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520583af1f264f22a86722b0cb9ff405'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:19.723147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1310/2625] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:19 2022] DELETE /identity/v3/users/520583af1f264f22a86722b0cb9ff405 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:19.938729 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:19.950113 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20faf66-e5fb-47bc-b460-3181b2a54f67 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.016368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.017200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.017479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.017757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] PATH_INFO: `/v3/projects/c160b9f06dad4ee98349dbc63131c20d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.019027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c160b9f06dad4ee98349dbc63131c20d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.021034 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.068014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:20.083701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c160b9f06dad4ee98349dbc63131c20d', 'request_id': 'req-7129c959-0d99-4eaa-ac71-affa91a14849'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:20.098015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c160b9f06dad4ee98349dbc63131c20d', 'request_id': 'req-7129c959-0d99-4eaa-ac71-affa91a14849'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:20.107531 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c160b9f06dad4ee98349dbc63131c20d', 'request_id': 'req-7129c959-0d99-4eaa-ac71-affa91a14849'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:20.119577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] The token cache is being invalidate because project c160b9f06dad4ee98349dbc63131c20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:20.119865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:20.120572 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7129c959-0d99-4eaa-ac71-affa91a14849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:20.122168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1316/2626] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:19 2022] DELETE /identity/v3/projects/c160b9f06dad4ee98349dbc63131c20d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:20.127826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.128116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.128425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.147525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:20.148820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:20.213979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1311/2627] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:20.222489 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.222782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.223065 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.242638 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:20.244108 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:20.252724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1317/2628] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:20.257150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.268163 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec5cdc-7f45-4ee4-9903-7271086d3080 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.272828 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.273514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.273800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.274082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.274836 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:20.275143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.276811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.278568 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:20.292874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1312/2629] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:08:20.297430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.307199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0c52-9f0a-4a9e-bf84-06f07d023412 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.311951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.312663 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.312950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.313230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.313942 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.315939 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.343381 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1318/2630] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:20 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:08:20.347946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.358199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a022fef-cca1-4375-8e87-eed81b532647 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.425261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.426025 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.426485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.426943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.428180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.431630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.461594 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1313/2631] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:20 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:08:20.465995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.478214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bf120-d159-43af-a300-b27c7c447dfa admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.489142 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.489830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.490118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.490567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.491705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.493013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.504940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1319/2632] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:20.509086 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.521624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7b08-a164-4e4c-8d7a-0a10806df423 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.526503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e4d3c-9381-4e78-b2b3-6a17cef171e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.527182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4d3c-9381-4e78-b2b3-6a17cef171e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.527784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4d3c-9381-4e78-b2b3-6a17cef171e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.527784 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4d3c-9381-4e78-b2b3-6a17cef171e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.528486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e4d3c-9381-4e78-b2b3-6a17cef171e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.530155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e4d3c-9381-4e78-b2b3-6a17cef171e0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.542352 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1314/2633] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:20.546759 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.557252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd3bbf-478b-4e3b-b8ad-82607c460768 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.562601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.563514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.563823 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.564087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] PATH_INFO: `/v3/projects/0005c811375d4a55bb5bc2844103d24e/users/c8dbb3b553a84b07b2d407af42116375/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.581173 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0005c811375d4a55bb5bc2844103d24e, user_id=c8dbb3b553a84b07b2d407af42116375, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.583866 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.598625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1320/2634] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:20 2022] PUT /identity/v3/projects/0005c811375d4a55bb5bc2844103d24e/users/c8dbb3b553a84b07b2d407af42116375/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:20.609447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.610666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.610666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.644080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] MFA Rules not processed for user `c8dbb3b553a84b07b2d407af42116375`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:20.645515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:20.722344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1315/2635] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:20.733507 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:20.735536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:20.835503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86a130-5e1e-4c3d-9439-ed84d5cb23b3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:20.847912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b992ec-ea42-425a-818f-bfed6972114c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:20.849007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b992ec-ea42-425a-818f-bfed6972114c service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:20.849292 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b992ec-ea42-425a-818f-bfed6972114c service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:20.849587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b992ec-ea42-425a-818f-bfed6972114c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:20.859987 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b992ec-ea42-425a-818f-bfed6972114c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:20.862193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b992ec-ea42-425a-818f-bfed6972114c service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:20.874152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1321/2636] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:08:28.236649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:28.303439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efacd74-db23-49d7-ba47-4d02ee9fc279 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:28.308115 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:28.309071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:28.309351 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:28.309666 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] PATH_INFO: `/v3/users/c8dbb3b553a84b07b2d407af42116375` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:28.310938 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8dbb3b553a84b07b2d407af42116375)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:28.313058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:28.316222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:28.373312 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8dbb3b553a84b07b2d407af42116375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:28.387844 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8dbb3b553a84b07b2d407af42116375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:28.399652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8dbb3b553a84b07b2d407af42116375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:28.408755 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8dbb3b553a84b07b2d407af42116375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:28.409367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8dbb3b553a84b07b2d407af42116375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:28.415662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1316/2637] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:28 2022] DELETE /identity/v3/users/c8dbb3b553a84b07b2d407af42116375 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:28.420114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b68c-17f8-463b-bde3-ccfd2c590730 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:28.420652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b68c-17f8-463b-bde3-ccfd2c590730 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:28.420991 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166b68c-17f8-463b-bde3-ccfd2c590730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:28.440977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4166b68c-17f8-463b-bde3-ccfd2c590730 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:28.442342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4166b68c-17f8-463b-bde3-ccfd2c590730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:28.486857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:28.508998 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1322/2638] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:28.519439 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:28.521002 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:28.584239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832455e0-93d7-4db1-85ce-a66a83e012ad admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:28.589884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:28.590580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:28.590868 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:28.591306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:28.602225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:28.603668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:28.608598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:28.608893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:28.609317 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:28.614117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1317/2639] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:28.629139 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] MFA Rules not processed for user `4a3a211465a0478994d011f89eaf972f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:28.630362 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:28.690288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1323/2640] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:28.912379 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:28.914396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:28.929020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb32774-9664-4bb7-9d5f-ae3eb9886baf service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:28.933915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:28.935695 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:28.936566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:28.936566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] PATH_INFO: `/v3/projects/0005c811375d4a55bb5bc2844103d24e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:28.937727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0005c811375d4a55bb5bc2844103d24e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:28.940049 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:28.992595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0005c811375d4a55bb5bc2844103d24e', 'request_id': 'req-78efe424-b49d-4dd0-8407-25d445f837ca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:28.999485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0005c811375d4a55bb5bc2844103d24e', 'request_id': 'req-78efe424-b49d-4dd0-8407-25d445f837ca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:29.007447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0005c811375d4a55bb5bc2844103d24e', 'request_id': 'req-78efe424-b49d-4dd0-8407-25d445f837ca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:29.019629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.021721 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] The token cache is being invalidate because project 0005c811375d4a55bb5bc2844103d24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:29.022143 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:29.022860 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-78efe424-b49d-4dd0-8407-25d445f837ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:29.024534 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1318/2641] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:28 2022] DELETE /identity/v3/projects/0005c811375d4a55bb5bc2844103d24e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:29.029362 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.029656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.029952 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.032026 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd959bbc-726b-466f-ae5c-5031fdea935b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.045376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:29.051837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:29.053665 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:29.086191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.086737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.087027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.087304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.089257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:29.099615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.101130 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff0a37-e97f-445a-8340-c1de6587fe26 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.109889 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1324/2642] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:29.117855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1319/2643] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:08:29.126183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.126479 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.126899 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.147446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:29.147446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:29.154465 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1325/2644] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:29.158735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.168794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9086912-a419-4f2d-b4cf-8e54d05cdd50 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.174030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.174701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.174982 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.175316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.176027 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:29.176331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.178371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.193110 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1320/2645] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:08:29.197557 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.207406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26123bf8-d2c0-4625-a137-ef57f2912670 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.212323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.213022 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.213306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.213584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.214308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.216244 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.247131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1326/2646] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:29.248957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.261375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4e258-4c5d-41a1-8739-9f9dabae0ed9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.331512 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.332182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.332493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.332856 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.333460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.335358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.357981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1321/2647] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:29 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:29.362311 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.372444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcec68e-fa47-4182-a916-b536d326219c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.377961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.378624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.378905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.379181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.379902 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.381545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.391714 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1327/2648] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:08:29.395818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.406555 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048504-fcfc-4f59-95e5-4decc6bad6d6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.411703 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3337e9d-597d-48ff-9328-83b8284ddf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.412528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3337e9d-597d-48ff-9328-83b8284ddf52 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.412822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3337e9d-597d-48ff-9328-83b8284ddf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.413107 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3337e9d-597d-48ff-9328-83b8284ddf52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.413848 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3337e9d-597d-48ff-9328-83b8284ddf52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.415484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3337e9d-597d-48ff-9328-83b8284ddf52 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.431042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1322/2649] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:29.436655 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.450446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f3eb7-6cc4-4713-992d-6570d95a154f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.455721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.456788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.457093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.457473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] PATH_INFO: `/v3/projects/c5881caf60e042a0badab3d6dba4da7b/users/eb3572c2dece45abbba432dc286d7503/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.476995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5881caf60e042a0badab3d6dba4da7b, user_id=eb3572c2dece45abbba432dc286d7503, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.479807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.496066 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1328/2650] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:29 2022] PUT /identity/v3/projects/c5881caf60e042a0badab3d6dba4da7b/users/eb3572c2dece45abbba432dc286d7503/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:29.506387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.506793 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.507175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.539914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] MFA Rules not processed for user `eb3572c2dece45abbba432dc286d7503`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:29.541175 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:29.609813 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1323/2651] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:29.621107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.622819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:29.656770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:29.704674 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ceeabc-65c4-45b1-95f3-07201b4d4b57 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:29.710632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b38f6-e664-4217-bcb8-bf24e6ad1863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:29.711368 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b38f6-e664-4217-bcb8-bf24e6ad1863 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:29.711656 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b38f6-e664-4217-bcb8-bf24e6ad1863 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:29.711936 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b38f6-e664-4217-bcb8-bf24e6ad1863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:29.720647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b38f6-e664-4217-bcb8-bf24e6ad1863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:29.722430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b38f6-e664-4217-bcb8-bf24e6ad1863 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:29.733061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1329/2652] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:29.979548 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:29.982920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:30.046083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0879170-a32c-4812-af90-d2ea2126b7e3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:30.051397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:30.052267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:30.052579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:30.052859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] PATH_INFO: `/v3/users/eb3572c2dece45abbba432dc286d7503` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:30.054070 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb3572c2dece45abbba432dc286d7503)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:30.056055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:30.108633 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb3572c2dece45abbba432dc286d7503'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.120584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb3572c2dece45abbba432dc286d7503'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.128546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb3572c2dece45abbba432dc286d7503'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.135939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb3572c2dece45abbba432dc286d7503'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.136596 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb3572c2dece45abbba432dc286d7503'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.143063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1324/2653] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:29 2022] DELETE /identity/v3/users/eb3572c2dece45abbba432dc286d7503 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:30.147701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:30.147995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:30.148280 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:30.165622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:30.166818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:30.234364 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1330/2654] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:30.244544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:30.245999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:30.310906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ae7db-c8b3-4b85-b506-aae1e267efdf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:30.314639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b336a75-6204-47ca-8d96-e56bfd125e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:30.315291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b336a75-6204-47ca-8d96-e56bfd125e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:30.315567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b336a75-6204-47ca-8d96-e56bfd125e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:30.315832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b336a75-6204-47ca-8d96-e56bfd125e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:30.325550 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b336a75-6204-47ca-8d96-e56bfd125e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:30.327572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b336a75-6204-47ca-8d96-e56bfd125e55 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:30.338956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1325/2655] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:30.743262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:30.744919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:30.758914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce545cb4-f831-4825-bf81-4e438077ab15 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:30.763625 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:30.764578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:30.764890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:30.765196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] PATH_INFO: `/v3/projects/c5881caf60e042a0badab3d6dba4da7b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:30.766441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5881caf60e042a0badab3d6dba4da7b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:30.768582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:30.821708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5881caf60e042a0badab3d6dba4da7b', 'request_id': 'req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.829713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5881caf60e042a0badab3d6dba4da7b', 'request_id': 'req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.838772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5881caf60e042a0badab3d6dba4da7b', 'request_id': 'req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.850297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] The token cache is being invalidate because project c5881caf60e042a0badab3d6dba4da7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:30.850631 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.851227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3846c099-7ef8-47f0-ae8c-2b0d6b7211ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:30.852834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1331/2656] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:30 2022] DELETE /identity/v3/projects/c5881caf60e042a0badab3d6dba4da7b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:30.858441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:30.858778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:30.859063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:30.879251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:30.880368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:30.901135 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:30.948258 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1326/2657] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:30.957580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:30.957580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:30.958054 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:30.978053 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:30.979401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:30.992160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1332/2658] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:30.995882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.006732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f014b2-65d1-4584-ba2e-089b3402accb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.011823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.012580 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.013040 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.013339 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.014103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:31.014445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.016082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.030423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1327/2659] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:08:31.037385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.052520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e1a8d-7a1a-4d8c-a16f-2b4c3db3495b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.057712 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.058522 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.058905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.059304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.060124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.062158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.095843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1333/2660] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:31 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:31.100427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.112145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415c2bb-b107-4f80-89dd-692310a4ab8e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.195249 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.195981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.196260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.196565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.197319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.199300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.220500 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1328/2661] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:31 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:08:31.224929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.238183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1df29-de78-4ff1-a0e9-4cc9b079a183 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.243170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.243894 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.244175 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.244546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.245296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.246955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.249537 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:31.261256 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1334/2662] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:31.266745 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.277891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5c401-a320-41fa-9d35-512e788be68d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.283820 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f0baf-5c4a-4652-83eb-6ec62b4b5558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.284565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f0baf-5c4a-4652-83eb-6ec62b4b5558 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.284866 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f0baf-5c4a-4652-83eb-6ec62b4b5558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.285113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f0baf-5c4a-4652-83eb-6ec62b4b5558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.285872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f0baf-5c4a-4652-83eb-6ec62b4b5558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.288435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f0baf-5c4a-4652-83eb-6ec62b4b5558 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.301552 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1329/2663] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:31.306214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.320837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ea474-4562-437b-84af-68f5236e5f82 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.328935 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.329877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.330283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.330582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] PATH_INFO: `/v3/projects/d32a35ad018a466b81b6b960f16891ba/users/1d43b3f047a64820871ad2b72dccd65b/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.349644 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d32a35ad018a466b81b6b960f16891ba, user_id=1d43b3f047a64820871ad2b72dccd65b, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.352412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.373604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1335/2664] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:31 2022] PUT /identity/v3/projects/d32a35ad018a466b81b6b960f16891ba/users/1d43b3f047a64820871ad2b72dccd65b/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:31.383886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.384493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.384610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.416701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] MFA Rules not processed for user `1d43b3f047a64820871ad2b72dccd65b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:31.418768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:31.515368 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1330/2665] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:31.520090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.590119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaea87-d1ad-48e3-870b-523a033151a4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.595154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.595835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.596123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.596415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.597150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.599955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.631821 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1336/2666] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:31 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:31.633021 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.643719 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3d5bf-aa3c-4db9-a7d2-05039f1ffbc8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.661812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:31.710831 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.711517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.711791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.712064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.712887 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.714832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.742085 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1331/2667] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:31 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:08:31.743400 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.755366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd749-f456-4f45-bcdd-6d99091895d8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.761039 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.761742 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.762030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.762310 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.763096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.764781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.777873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1337/2668] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:08:31.782551 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.793507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fce633-6a3f-4a3a-9d2c-d7b41a8ebd83 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.801197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.802240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.802528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.802806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] PATH_INFO: `/v3/projects/a26983d78af74ffaaa4f93943ec0fa97/users/6e9491985b2d4f4083e3f6d2db4ff3e4/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.824528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26983d78af74ffaaa4f93943ec0fa97, user_id=6e9491985b2d4f4083e3f6d2db4ff3e4, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.827802 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.849712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1332/2669] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:31 2022] PUT /identity/v3/projects/a26983d78af74ffaaa4f93943ec0fa97/users/6e9491985b2d4f4083e3f6d2db4ff3e4/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:31.854160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.866773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af766c-f388-4620-a49e-f2f08f7ed8ec admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.941969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbee40-5d4b-4566-839a-509ded540fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.942533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbee40-5d4b-4566-839a-509ded540fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.942817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbee40-5d4b-4566-839a-509ded540fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.943093 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbee40-5d4b-4566-839a-509ded540fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.943834 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbee40-5d4b-4566-839a-509ded540fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.945554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbee40-5d4b-4566-839a-509ded540fe8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.960085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1338/2670] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:31.961648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:31.973354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268f5e9-d52b-473a-a957-4994d581e17b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:31.978199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:31.979087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:31.979373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:31.979654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] PATH_INFO: `/v3/projects/a26983d78af74ffaaa4f93943ec0fa97/users/6e9491985b2d4f4083e3f6d2db4ff3e4/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:31.981778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26983d78af74ffaaa4f93943ec0fa97, user_id=6e9491985b2d4f4083e3f6d2db4ff3e4, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:31.985149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:31.996434 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1333/2671] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:31 2022] PUT /identity/v3/projects/a26983d78af74ffaaa4f93943ec0fa97/users/6e9491985b2d4f4083e3f6d2db4ff3e4/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:32.008681 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:32.008973 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:32.009254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:32.039536 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] MFA Rules not processed for user `6e9491985b2d4f4083e3f6d2db4ff3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:32.040822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:32.076168 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:32.118940 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1339/2672] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:32.131344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:32.133437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:32.207922 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53105d-c753-4352-b1e3-a87359311cc1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:32.211889 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:32.212592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:32.212872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:32.213144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:32.223832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:32.225503 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:32.236345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1334/2673] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:32.332495 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:32.356688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39d6fa-7028-441c-99c0-ee71269e9750 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:32.368636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de4661-42e4-4115-9a64-5212cc3f776a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:32.370009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de4661-42e4-4115-9a64-5212cc3f776a service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:32.370406 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de4661-42e4-4115-9a64-5212cc3f776a service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:32.370863 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de4661-42e4-4115-9a64-5212cc3f776a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:32.447872 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de4661-42e4-4115-9a64-5212cc3f776a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:32.449391 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de4661-42e4-4115-9a64-5212cc3f776a service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:32.461806 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1340/2674] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:41.433313 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:41.500761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329563-ce77-43f2-bfb9-600b3595df21 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:41.505627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:41.506510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:41.506790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:41.507092 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] PATH_INFO: `/v3/users/1d43b3f047a64820871ad2b72dccd65b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:41.508473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d43b3f047a64820871ad2b72dccd65b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:41.510564 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:41.528045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:41.567497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d43b3f047a64820871ad2b72dccd65b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:41.581031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d43b3f047a64820871ad2b72dccd65b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:41.588566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d43b3f047a64820871ad2b72dccd65b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:41.596592 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d43b3f047a64820871ad2b72dccd65b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:41.597804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d43b3f047a64820871ad2b72dccd65b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:41.604319 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1335/2675] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:41 2022] DELETE /identity/v3/users/1d43b3f047a64820871ad2b72dccd65b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:41.609056 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:41.609582 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:41.609683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:41.627626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:41.628869 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:41.688590 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1341/2676] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:41.698288 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:41.699756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:41.765182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fc513-d895-434f-b689-212ecc49c373 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:41.768885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:41.769539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:41.769823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:41.770108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:41.779716 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:41.781205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:41.791965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1336/2677] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:42.096462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:42.098219 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:42.119454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389bba7-d25c-4334-8898-43fba646ab60 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:42.125275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:42.126155 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:42.126430 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:42.126745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] PATH_INFO: `/v3/projects/d32a35ad018a466b81b6b960f16891ba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:42.128047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d32a35ad018a466b81b6b960f16891ba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:42.130064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:42.187837 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd32a35ad018a466b81b6b960f16891ba', 'request_id': 'req-0a7a1034-b2b4-4abb-a17b-ef6396c09893'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.194807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd32a35ad018a466b81b6b960f16891ba', 'request_id': 'req-0a7a1034-b2b4-4abb-a17b-ef6396c09893'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.202995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd32a35ad018a466b81b6b960f16891ba', 'request_id': 'req-0a7a1034-b2b4-4abb-a17b-ef6396c09893'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.204955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:42.215439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] The token cache is being invalidate because project d32a35ad018a466b81b6b960f16891ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:42.215645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.216807 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.218539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1342/2678] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:42 2022] DELETE /identity/v3/projects/d32a35ad018a466b81b6b960f16891ba => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:42.222482 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:42.232902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f376ea-2662-49a7-8ed2-48b68ce3f78f service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:42.300299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:42.301205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:42.301493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:42.301770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] PATH_INFO: `/v3/users/6e9491985b2d4f4083e3f6d2db4ff3e4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:42.302974 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9491985b2d4f4083e3f6d2db4ff3e4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:42.304937 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:42.348080 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:42.356199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9491985b2d4f4083e3f6d2db4ff3e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.369359 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9491985b2d4f4083e3f6d2db4ff3e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.377083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9491985b2d4f4083e3f6d2db4ff3e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.386578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9491985b2d4f4083e3f6d2db4ff3e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.387300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6c63850-9751-4e7d-83f8-f1ed6d0965fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9491985b2d4f4083e3f6d2db4ff3e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.394558 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1337/2679] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:42 2022] DELETE /identity/v3/users/6e9491985b2d4f4083e3f6d2db4ff3e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:08:42.716550 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:42.718028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:42.731493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1034-b2b4-4abb-a17b-ef6396c09893 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:42.805580 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:42.806691 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:42.807055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:42.807431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] PATH_INFO: `/v3/projects/a26983d78af74ffaaa4f93943ec0fa97` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:42.809283 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26983d78af74ffaaa4f93943ec0fa97)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:42.811584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:42.865777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26983d78af74ffaaa4f93943ec0fa97', 'request_id': 'req-cff13d45-c193-4639-9997-7cf3eb058979'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.874635 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26983d78af74ffaaa4f93943ec0fa97', 'request_id': 'req-cff13d45-c193-4639-9997-7cf3eb058979'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.884413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26983d78af74ffaaa4f93943ec0fa97', 'request_id': 'req-cff13d45-c193-4639-9997-7cf3eb058979'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.898433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] The token cache is being invalidate because project a26983d78af74ffaaa4f93943ec0fa97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:42.898783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.899448 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cff13d45-c193-4639-9997-7cf3eb058979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:42.901296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1343/2680] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:42 2022] DELETE /identity/v3/projects/a26983d78af74ffaaa4f93943ec0fa97 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:42.907756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:42.908289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:42.908493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:42.930524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:42.931908 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:43.008279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1338/2681] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:08:43.017249 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.017546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.017843 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.039267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:43.041009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:43.050521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1344/2682] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:43.054365 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.065215 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334170-ab46-46cb-b489-e523d57fc738 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.071425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.072109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.072426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.072993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.073781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:43.074094 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.075833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.089710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1339/2683] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:08:43.094382 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.107432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef812f-3be2-41e6-821f-f30d407e70f5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.113429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.113429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.113713 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.113933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.114805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.116745 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.151799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1345/2684] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:43.156265 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.169102 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae3b85-8916-418c-8d22-e8cd92a91b14 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.252656 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.253071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.253570 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.254128 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.255084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.257553 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.281041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1340/2685] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:43.285258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.297246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c5cbf-30e7-47fd-9b2f-6d82dec49d82 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.301914 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.302585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.302867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.303177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.303915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.305545 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.317008 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1346/2686] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:43.321167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.332613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51c06-2c13-458d-a261-5a3fe76189cd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.337233 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19a18d-eb08-4137-81e4-3d607bc1ff70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.337907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19a18d-eb08-4137-81e4-3d607bc1ff70 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.338189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19a18d-eb08-4137-81e4-3d607bc1ff70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.338465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19a18d-eb08-4137-81e4-3d607bc1ff70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.339196 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19a18d-eb08-4137-81e4-3d607bc1ff70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.340823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19a18d-eb08-4137-81e4-3d607bc1ff70 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.352345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1341/2687] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:43.356651 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.367978 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b74a8-44f0-41ec-bbd7-5c1becd206a5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.373808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.374835 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.375218 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.375487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] PATH_INFO: `/v3/projects/d64e1e31621b4710bbf181d1d68c6e46/users/22568e140f9f437e83309e34b1b6e353/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.398308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64e1e31621b4710bbf181d1d68c6e46, user_id=22568e140f9f437e83309e34b1b6e353, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.401488 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.417223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1347/2688] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:43 2022] PUT /identity/v3/projects/d64e1e31621b4710bbf181d1d68c6e46/users/22568e140f9f437e83309e34b1b6e353/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:08:43.436634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.436927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.437238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.466855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] MFA Rules not processed for user `22568e140f9f437e83309e34b1b6e353`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:43.467941 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:43.477800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:43.546235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1342/2689] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:43.550801 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.573789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:43.621706 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868284b-2dcf-4287-b766-9e91c40267fd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.627044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.627795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.628182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.628502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.629220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.631133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.664009 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1348/2690] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:08:43.665158 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.676730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013ea2-a7fd-4a00-8e57-09de88764500 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.738639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.739451 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.739740 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.740011 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.740907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.742753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.764087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1343/2691] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:08:43.768141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.779135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d6e4-577d-4c59-9df9-fa23a5ce7527 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.783817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.784520 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.784812 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.785092 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.785884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.787503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.798312 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1349/2692] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:43.802566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.813507 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619a42f-8213-456d-99f6-f379d998457e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.818171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.819048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.819334 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.819612 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] PATH_INFO: `/v3/projects/2f29e72a9e38421e9590fca8cb7e5679/users/d370881ca5ea4a12bca48c8b8a8e9bdb/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.834920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f29e72a9e38421e9590fca8cb7e5679, user_id=d370881ca5ea4a12bca48c8b8a8e9bdb, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.837618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.850761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1344/2693] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:43 2022] PUT /identity/v3/projects/2f29e72a9e38421e9590fca8cb7e5679/users/d370881ca5ea4a12bca48c8b8a8e9bdb/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:43.854941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.865366 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6447-6d3e-4dfb-a9c5-9bb11dfb6f0d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.927506 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afda8b5-889b-4341-8d6f-bbf97ca0285a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.928185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afda8b5-889b-4341-8d6f-bbf97ca0285a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.928514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afda8b5-889b-4341-8d6f-bbf97ca0285a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.928795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afda8b5-889b-4341-8d6f-bbf97ca0285a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.929546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afda8b5-889b-4341-8d6f-bbf97ca0285a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.931153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afda8b5-889b-4341-8d6f-bbf97ca0285a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.942136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1350/2694] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:43.947031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:43.958278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a28777-7148-4aef-bbdb-a3147dcb9208 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:43.963033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:43.963942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.964217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.964697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] PATH_INFO: `/v3/projects/2f29e72a9e38421e9590fca8cb7e5679/users/d370881ca5ea4a12bca48c8b8a8e9bdb/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:43.966601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f29e72a9e38421e9590fca8cb7e5679, user_id=d370881ca5ea4a12bca48c8b8a8e9bdb, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:43.969285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:43.980259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1345/2695] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:43 2022] PUT /identity/v3/projects/2f29e72a9e38421e9590fca8cb7e5679/users/d370881ca5ea4a12bca48c8b8a8e9bdb/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:43.992352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:43.992587 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:43.992877 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:44.024567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] MFA Rules not processed for user `d370881ca5ea4a12bca48c8b8a8e9bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:44.025805 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:44.098248 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1351/2696] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:44.118347 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:44.120160 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:44.182016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dcec5-9eeb-41ab-bd64-e7971178738e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:44.186531 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b456e2-e7c0-417a-82fc-e8afa76cc112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:44.187209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b456e2-e7c0-417a-82fc-e8afa76cc112 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:44.187493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b456e2-e7c0-417a-82fc-e8afa76cc112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:44.187773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b456e2-e7c0-417a-82fc-e8afa76cc112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:44.198234 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b456e2-e7c0-417a-82fc-e8afa76cc112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:44.199674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b456e2-e7c0-417a-82fc-e8afa76cc112 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:44.209826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1346/2697] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:44.601331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:44.616426 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:44.666466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42110603-e69e-47b4-b379-2d3d4efb81c6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:44.672105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:44.673061 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:44.673344 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:44.673620 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] PATH_INFO: `/v3/users/22568e140f9f437e83309e34b1b6e353` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:44.674932 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22568e140f9f437e83309e34b1b6e353)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:44.676933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:44.727028 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22568e140f9f437e83309e34b1b6e353'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:44.739969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22568e140f9f437e83309e34b1b6e353'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:44.748020 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22568e140f9f437e83309e34b1b6e353'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:44.748683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22568e140f9f437e83309e34b1b6e353'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:44.749301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22568e140f9f437e83309e34b1b6e353'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:44.761937 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1352/2698] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:44 2022] DELETE /identity/v3/users/22568e140f9f437e83309e34b1b6e353 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:44.766940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:44.767230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:44.767537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:44.780137 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:44.786178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:44.787801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:44.846651 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1347/2699] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:44.856297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:44.857819 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:44.918598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1f89c-99d4-4515-883f-d117ac4edf51 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:44.922227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:44.922885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:44.923171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:44.923453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:44.932885 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:44.934324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:44.944094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1353/2700] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:45.238185 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:45.240250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:45.267480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9189c3-6c8e-4322-ba18-b1c30803cb9c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:45.272691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:45.274442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:45.274833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:45.274997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] PATH_INFO: `/v3/projects/d64e1e31621b4710bbf181d1d68c6e46` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:45.276300 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64e1e31621b4710bbf181d1d68c6e46)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:45.278239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:45.330238 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64e1e31621b4710bbf181d1d68c6e46', 'request_id': 'req-d09eb8be-eff0-4d31-b15f-a13d94508598'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.337308 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64e1e31621b4710bbf181d1d68c6e46', 'request_id': 'req-d09eb8be-eff0-4d31-b15f-a13d94508598'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.345029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64e1e31621b4710bbf181d1d68c6e46', 'request_id': 'req-d09eb8be-eff0-4d31-b15f-a13d94508598'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.357189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] The token cache is being invalidate because project d64e1e31621b4710bbf181d1d68c6e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:45.357525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.358138 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.359699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1348/2701] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:45 2022] DELETE /identity/v3/projects/d64e1e31621b4710bbf181d1d68c6e46 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:08:45.364440 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:45.375514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400844e1-e800-4a64-9448-d47e3c7a8648 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:45.409157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:45.437343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:45.438222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:45.438503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:45.438780 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] PATH_INFO: `/v3/users/d370881ca5ea4a12bca48c8b8a8e9bdb` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:45.439971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d370881ca5ea4a12bca48c8b8a8e9bdb)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:45.441933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:45.499476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd370881ca5ea4a12bca48c8b8a8e9bdb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.512029 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd370881ca5ea4a12bca48c8b8a8e9bdb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.520668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd370881ca5ea4a12bca48c8b8a8e9bdb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.521308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd370881ca5ea4a12bca48c8b8a8e9bdb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.521911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74f49711-cde5-474b-a862-1004d3b274fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd370881ca5ea4a12bca48c8b8a8e9bdb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.534930 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1354/2702] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:45 2022] DELETE /identity/v3/users/d370881ca5ea4a12bca48c8b8a8e9bdb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:45.814336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:45.815853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:45.830200 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09eb8be-eff0-4d31-b15f-a13d94508598 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:45.888914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:45.889872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:45.890181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:45.890505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] PATH_INFO: `/v3/projects/2f29e72a9e38421e9590fca8cb7e5679` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:45.891734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f29e72a9e38421e9590fca8cb7e5679)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:45.893785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:45.931935 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:45.946458 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f29e72a9e38421e9590fca8cb7e5679', 'request_id': 'req-76128b4d-213a-4914-9305-dc3d71cadc05'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.952918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f29e72a9e38421e9590fca8cb7e5679', 'request_id': 'req-76128b4d-213a-4914-9305-dc3d71cadc05'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.960271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f29e72a9e38421e9590fca8cb7e5679', 'request_id': 'req-76128b4d-213a-4914-9305-dc3d71cadc05'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.970943 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] The token cache is being invalidate because project 2f29e72a9e38421e9590fca8cb7e5679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:45.971230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.971853 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76128b4d-213a-4914-9305-dc3d71cadc05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:45.973425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1349/2703] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:45 2022] DELETE /identity/v3/projects/2f29e72a9e38421e9590fca8cb7e5679 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:45.978436 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:45.978943 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:45.979046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:45.998668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:45.999961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:46.059989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1355/2704] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:46.067797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.068090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.068403 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.087546 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:46.088792 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:46.097589 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1350/2705] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:46.101906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.112761 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3409c03-e4e4-4719-941e-c321cc6f9b07 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.120265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.120962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.121246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.121521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.122252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:46.122600 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.124199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.136011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1356/2706] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:08:46.140345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.149948 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873da4df-7d24-4d57-afc6-c7d58a0a16f6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.155209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.155865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.156147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.156452 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.157146 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.159050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.186606 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1351/2707] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:46.190704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.200716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56167b59-a0ad-40cd-b180-39723b412bcd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.240769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:46.258431 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.259096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.259398 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.259671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.260353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.262417 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.282967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1357/2708] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:46.287395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.297783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc013c-1f23-4e1d-8d10-2e9f1e0af401 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.302330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.302984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.303267 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.303543 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.304286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.305890 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.316688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1352/2709] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:46.322669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.334216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159db3ad-9557-48c5-b02c-1d42517c3b11 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.338833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825e2ba-699f-4513-9611-6379d86f6902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.339500 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825e2ba-699f-4513-9611-6379d86f6902 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.339789 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825e2ba-699f-4513-9611-6379d86f6902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.340070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825e2ba-699f-4513-9611-6379d86f6902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.340846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825e2ba-699f-4513-9611-6379d86f6902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.343060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825e2ba-699f-4513-9611-6379d86f6902 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.353799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1358/2710] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:46.358268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.368627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd87dba-dfdd-47bb-aef0-e6746e1bfaf6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.373280 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.374164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.374448 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.374730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] PATH_INFO: `/v3/projects/0d156cdfa9dd4b5785d6d76b0f270875/users/0b76ac38bb7e469c9b4d350efab95885/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.389749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d156cdfa9dd4b5785d6d76b0f270875, user_id=0b76ac38bb7e469c9b4d350efab95885, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.392460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.407195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1353/2711] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:46 2022] PUT /identity/v3/projects/0d156cdfa9dd4b5785d6d76b0f270875/users/0b76ac38bb7e469c9b4d350efab95885/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:46.417871 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.418196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.418481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.444394 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] MFA Rules not processed for user `0b76ac38bb7e469c9b4d350efab95885`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:46.445632 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:46.512762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1359/2712] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:46.517939 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.579112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae848f-f46d-49a9-8e86-65bc7fda5a66 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.583718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.584408 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.584709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.584988 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.585690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.587609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.614081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1354/2713] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:46.618324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.629723 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21937bb2-b568-41cb-b0f7-429837466175 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.687085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.687768 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.688058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.688342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.689152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.691120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.715146 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1360/2714] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:46.720260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.730762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195ca1-c69d-4867-9720-1bfb1c81db1c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.735697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.736367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.736690 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.736965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.737707 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.739304 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.751007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1355/2715] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:46.755325 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.766512 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545b327-4b26-4864-b9f9-f5ad3bfde068 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.771324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.772227 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.772603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.772888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] PATH_INFO: `/v3/projects/eba0d554320445ba9dc5e45223b51d81/users/e32576ecf656401e9e5d7dd917f3337a/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.788413 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba0d554320445ba9dc5e45223b51d81, user_id=e32576ecf656401e9e5d7dd917f3337a, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.791094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.805323 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1361/2716] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:46 2022] PUT /identity/v3/projects/eba0d554320445ba9dc5e45223b51d81/users/e32576ecf656401e9e5d7dd917f3337a/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:46.809392 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.820445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60aaae-54a0-43aa-bba6-d4687b59207d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.849333 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:46.879222 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5459b-340c-456e-8081-c0323314c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.879895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5459b-340c-456e-8081-c0323314c4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.880180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5459b-340c-456e-8081-c0323314c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.880498 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5459b-340c-456e-8081-c0323314c4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.881232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5459b-340c-456e-8081-c0323314c4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.882826 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5459b-340c-456e-8081-c0323314c4a2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.893228 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1356/2717] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:46.897547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:46.907735 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd62a2-6f8e-4aef-9339-670c8ccdc4b2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:46.912684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:46.913575 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.913858 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.914135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] PATH_INFO: `/v3/projects/eba0d554320445ba9dc5e45223b51d81/users/e32576ecf656401e9e5d7dd917f3337a/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.916145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba0d554320445ba9dc5e45223b51d81, user_id=e32576ecf656401e9e5d7dd917f3337a, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:46.918798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:46.929195 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1362/2718] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:46 2022] PUT /identity/v3/projects/eba0d554320445ba9dc5e45223b51d81/users/e32576ecf656401e9e5d7dd917f3337a/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:46.941203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:46.941497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:46.941779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:46.969274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] MFA Rules not processed for user `e32576ecf656401e9e5d7dd917f3337a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:46.970526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:47.037356 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1357/2719] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:47.049163 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:47.050893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:47.056235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:47.110612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a251c6-8e7d-4062-b234-6849c1ec5878 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:47.114279 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbdec1-db48-41fd-83c4-4594302056af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:47.114926 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbdec1-db48-41fd-83c4-4594302056af service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:47.115212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbdec1-db48-41fd-83c4-4594302056af service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:47.115490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbdec1-db48-41fd-83c4-4594302056af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:47.125355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbdec1-db48-41fd-83c4-4594302056af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:47.126790 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbdec1-db48-41fd-83c4-4594302056af service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:47.136267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1363/2720] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:47.546179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:47.609098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7604a-002a-4252-ad7b-d1e9d1554ab4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:47.613927 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:47.614812 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:47.615099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:47.615382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] PATH_INFO: `/v3/users/0b76ac38bb7e469c9b4d350efab95885` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:47.616705 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b76ac38bb7e469c9b4d350efab95885)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:47.618646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:47.665598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b76ac38bb7e469c9b4d350efab95885'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:47.671268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:47.679576 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b76ac38bb7e469c9b4d350efab95885'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:47.686573 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b76ac38bb7e469c9b4d350efab95885'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:47.694100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b76ac38bb7e469c9b4d350efab95885'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:47.694808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b76ac38bb7e469c9b4d350efab95885'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:47.700895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1358/2721] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:47 2022] DELETE /identity/v3/users/0b76ac38bb7e469c9b4d350efab95885 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:47.707017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:47.707361 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:47.707707 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:47.731338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:47.733095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:47.794122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1364/2722] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:47.803760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:47.805272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:47.864399 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e639a5-8e27-4770-8bc2-16eb62a50fe1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:47.868191 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:47.868873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:47.869152 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:47.869427 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:47.879733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:47.881221 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:47.892081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1359/2723] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:48.169776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:48.171246 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:48.189132 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a72c3-c8a1-4578-a4df-e11335f11304 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:48.193753 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:48.194730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:48.194895 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:48.195273 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] PATH_INFO: `/v3/projects/0d156cdfa9dd4b5785d6d76b0f270875` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:48.196604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d156cdfa9dd4b5785d6d76b0f270875)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:48.198954 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:48.249882 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d156cdfa9dd4b5785d6d76b0f270875', 'request_id': 'req-3a793743-ebd2-44e2-af18-60b4628e17a8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.256309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d156cdfa9dd4b5785d6d76b0f270875', 'request_id': 'req-3a793743-ebd2-44e2-af18-60b4628e17a8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.264986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d156cdfa9dd4b5785d6d76b0f270875', 'request_id': 'req-3a793743-ebd2-44e2-af18-60b4628e17a8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.275564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] The token cache is being invalidate because project 0d156cdfa9dd4b5785d6d76b0f270875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:48.275854 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.276517 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.278098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1365/2724] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:48 2022] DELETE /identity/v3/projects/0d156cdfa9dd4b5785d6d76b0f270875 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:48.282142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:48.291839 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596b26b-4a3e-418b-9a35-1cbf7083f692 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:48.347878 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:48.349082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:48.349082 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:48.349320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] PATH_INFO: `/v3/users/e32576ecf656401e9e5d7dd917f3337a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:48.350505 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32576ecf656401e9e5d7dd917f3337a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:48.352455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:48.399971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32576ecf656401e9e5d7dd917f3337a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.411732 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32576ecf656401e9e5d7dd917f3337a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.418587 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32576ecf656401e9e5d7dd917f3337a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.421105 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:48.426345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32576ecf656401e9e5d7dd917f3337a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.426885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-601a8936-48dd-4d24-8a8b-a55e818fb22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32576ecf656401e9e5d7dd917f3337a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.433956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1360/2725] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:48 2022] DELETE /identity/v3/users/e32576ecf656401e9e5d7dd917f3337a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:48.713252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:48.714778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:48.729044 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793743-ebd2-44e2-af18-60b4628e17a8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:48.760474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:48.793873 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:48.794787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:48.795072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:48.795351 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] PATH_INFO: `/v3/projects/eba0d554320445ba9dc5e45223b51d81` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:48.796588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba0d554320445ba9dc5e45223b51d81)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:48.798503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:48.845073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba0d554320445ba9dc5e45223b51d81', 'request_id': 'req-40f53cbe-cb16-4787-81c7-2fec8e467d64'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.851212 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba0d554320445ba9dc5e45223b51d81', 'request_id': 'req-40f53cbe-cb16-4787-81c7-2fec8e467d64'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.858661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba0d554320445ba9dc5e45223b51d81', 'request_id': 'req-40f53cbe-cb16-4787-81c7-2fec8e467d64'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.869427 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] The token cache is being invalidate because project eba0d554320445ba9dc5e45223b51d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:48.869715 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.870363 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40f53cbe-cb16-4787-81c7-2fec8e467d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:48.871908 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1366/2726] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:48 2022] DELETE /identity/v3/projects/eba0d554320445ba9dc5e45223b51d81 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:48.877395 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:48.877858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:48.877984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:48.897363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:48.898565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:48.957387 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1361/2727] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:48.965253 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:48.965546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:48.965830 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:48.983980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:48.985223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:48.994328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1367/2728] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:48.998270 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.008537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0f6ae-c22a-4b11-bac3-c9a714c063e7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.014046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.014735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.015018 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.015291 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.016032 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:49.016337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.018001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.030923 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1362/2729] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:08:49.035696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.045688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28095ee3-3bfd-4af2-844a-c1bda29025d4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.050509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.051190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.051477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.051758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.052521 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.054458 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.086099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1368/2730] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:49 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:49.090554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.101020 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b160f3c-899b-4570-a24f-7e4f3df1eb49 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.167969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.168768 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.169207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.169524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.170375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.172355 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.193981 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1363/2731] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:49 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:49.198152 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.209228 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95117304-0f78-45d9-b88e-513ac5a7cb00 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.214104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.214779 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.215064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.215342 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.216105 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.217762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.228762 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1369/2732] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:49.233048 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.243547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc994c78-4b87-40bc-931f-c4dceed6b70b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.249023 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b024e-3a7d-4661-8c52-a798f3ae5573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.249821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b024e-3a7d-4661-8c52-a798f3ae5573 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.250101 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b024e-3a7d-4661-8c52-a798f3ae5573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.251281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b024e-3a7d-4661-8c52-a798f3ae5573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.251281 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b024e-3a7d-4661-8c52-a798f3ae5573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.255537 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b024e-3a7d-4661-8c52-a798f3ae5573 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.267329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1364/2733] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:49.271679 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.283696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e68b5-6188-415f-8ae0-e1ae529b9a48 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.288588 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.289499 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.289787 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.290070 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] PATH_INFO: `/v3/projects/3f5e77259c3d402fb6d0b2484c5419ae/users/3e86fdb71653424199a8cf2c20ae2339/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.307332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5e77259c3d402fb6d0b2484c5419ae, user_id=3e86fdb71653424199a8cf2c20ae2339, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.310721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.328493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1370/2734] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:49 2022] PUT /identity/v3/projects/3f5e77259c3d402fb6d0b2484c5419ae/users/3e86fdb71653424199a8cf2c20ae2339/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:49.339130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.339411 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.339691 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.367906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] MFA Rules not processed for user `3e86fdb71653424199a8cf2c20ae2339`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:49.369213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:49.418817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:49.439679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1365/2735] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:49.444880 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.504665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:49.510680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b1dc0-9211-41b2-aca8-7b09520e0465 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.516262 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.516950 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.517236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.517564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.518213 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.520115 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.545859 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1371/2736] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:49 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:49.549963 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.561193 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59caa7b-d50a-442d-b24e-0647808bba45 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.617400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.618063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.618345 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.618620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.619316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.621271 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.641786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1366/2737] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:49 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:49.645876 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.656842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8b21-37dc-4697-aeae-99237c8dc350 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.661909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.662576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.662865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.663145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.663875 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.665639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.677126 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1372/2738] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:49.681329 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.691363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018849-1fb2-436f-8151-9d8042e95d29 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.695915 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.696961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.697242 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.697517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] PATH_INFO: `/v3/projects/40ab9357783b419a83e192123b581543/users/ae0c8c7b3b2049ea96a1a65dc05d28c4/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.711874 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ab9357783b419a83e192123b581543, user_id=ae0c8c7b3b2049ea96a1a65dc05d28c4, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.714554 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.727997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1367/2739] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:49 2022] PUT /identity/v3/projects/40ab9357783b419a83e192123b581543/users/ae0c8c7b3b2049ea96a1a65dc05d28c4/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:49.732107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.743084 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa285a1-2691-46f5-bcc0-f0544715a0f3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.800073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dbaa6-2a68-4746-b9d3-9e0a26c23a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.800685 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dbaa6-2a68-4746-b9d3-9e0a26c23a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.800972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dbaa6-2a68-4746-b9d3-9e0a26c23a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.801252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dbaa6-2a68-4746-b9d3-9e0a26c23a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.801979 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dbaa6-2a68-4746-b9d3-9e0a26c23a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.803568 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dbaa6-2a68-4746-b9d3-9e0a26c23a0d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.814078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1373/2740] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:49.818456 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:49.828778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180c92c-4e36-4356-bed7-e60c10b65941 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:49.833383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:49.834268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.834547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.834824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] PATH_INFO: `/v3/projects/40ab9357783b419a83e192123b581543/users/ae0c8c7b3b2049ea96a1a65dc05d28c4/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.836873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ab9357783b419a83e192123b581543, user_id=ae0c8c7b3b2049ea96a1a65dc05d28c4, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:49.839497 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:49.850628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1368/2741] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:49 2022] PUT /identity/v3/projects/40ab9357783b419a83e192123b581543/users/ae0c8c7b3b2049ea96a1a65dc05d28c4/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:49.917803 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:49.918090 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:49.918401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:49.944783 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] MFA Rules not processed for user `ae0c8c7b3b2049ea96a1a65dc05d28c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:49.945985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:50.016647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1374/2742] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:50.027491 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:50.030182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:50.087832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c77170-b0b5-4aa5-931e-a885ae4c10c7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:50.092106 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:50.092780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:50.093064 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:50.093341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:50.102382 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:50.103811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:50.113462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1369/2743] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:50.211647 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:50.222365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa533ec-aee4-4e2b-9ee7-a7a752a6ec35 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:50.227490 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba93455-89ea-4327-aebd-4b1ea6a8b560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:50.228154 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba93455-89ea-4327-aebd-4b1ea6a8b560 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:50.228475 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba93455-89ea-4327-aebd-4b1ea6a8b560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:50.228763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba93455-89ea-4327-aebd-4b1ea6a8b560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:50.279373 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:50.290450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba93455-89ea-4327-aebd-4b1ea6a8b560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:50.291905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba93455-89ea-4327-aebd-4b1ea6a8b560 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:50.301548 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1375/2744] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:52.897112 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:52.962525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:53.028250 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7be6f-8b22-40f1-a1d2-2c66699c34e9 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:53.033087 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:53.033979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:53.034261 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:53.034542 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] PATH_INFO: `/v3/users/3e86fdb71653424199a8cf2c20ae2339` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:53.035808 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e86fdb71653424199a8cf2c20ae2339)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:53.037807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:53.097056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e86fdb71653424199a8cf2c20ae2339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.113206 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e86fdb71653424199a8cf2c20ae2339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.121527 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e86fdb71653424199a8cf2c20ae2339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.131264 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e86fdb71653424199a8cf2c20ae2339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.132109 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e86fdb71653424199a8cf2c20ae2339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.140810 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1370/2745] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:52 2022] DELETE /identity/v3/users/3e86fdb71653424199a8cf2c20ae2339 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:53.147254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:53.147814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:53.148075 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:53.207699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:53.208982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:53.274106 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1376/2746] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:53.286232 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:53.287724 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:53.368508 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331c62-1bc5-4c6a-86e8-b92109f3f812 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:53.372936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:53.373607 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:53.373894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:53.374174 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:53.384380 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:53.385967 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:53.404279 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1371/2747] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:53.724546 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:53.726308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:53.741231 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce8a24-f88c-4586-b223-e4c185093c2b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:53.746757 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:53.747836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:53.748127 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:53.748442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] PATH_INFO: `/v3/projects/3f5e77259c3d402fb6d0b2484c5419ae` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:53.749720 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5e77259c3d402fb6d0b2484c5419ae)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:53.751658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:53.804074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5e77259c3d402fb6d0b2484c5419ae', 'request_id': 'req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.810703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5e77259c3d402fb6d0b2484c5419ae', 'request_id': 'req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.819069 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5e77259c3d402fb6d0b2484c5419ae', 'request_id': 'req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.834929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] The token cache is being invalidate because project 3f5e77259c3d402fb6d0b2484c5419ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:53.835208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.835867 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.837591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1377/2748] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:53 2022] DELETE /identity/v3/projects/3f5e77259c3d402fb6d0b2484c5419ae => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:53.842323 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:53.852374 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ebd9e-898e-4b48-96c1-66223f8a866b service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:53.857134 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:53.913260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:53.914150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:53.914425 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:53.914829 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] PATH_INFO: `/v3/users/ae0c8c7b3b2049ea96a1a65dc05d28c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:53.915973 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0c8c7b3b2049ea96a1a65dc05d28c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:53.917929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:53.970031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0c8c7b3b2049ea96a1a65dc05d28c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.982898 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0c8c7b3b2049ea96a1a65dc05d28c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:53.992760 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0c8c7b3b2049ea96a1a65dc05d28c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.001445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0c8c7b3b2049ea96a1a65dc05d28c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.002103 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-055c918a-d354-49fb-b2ae-8a2f0205e859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0c8c7b3b2049ea96a1a65dc05d28c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.008700 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1372/2749] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:53 2022] DELETE /identity/v3/users/ae0c8c7b3b2049ea96a1a65dc05d28c4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:54.311274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.312825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:54.326947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0ae6b-0247-4e7b-bece-8f3bc6066986 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.384016 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:54.400266 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.401199 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.401481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.401758 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] PATH_INFO: `/v3/projects/40ab9357783b419a83e192123b581543` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.403049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ab9357783b419a83e192123b581543)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:54.405147 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:54.461985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ab9357783b419a83e192123b581543', 'request_id': 'req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.469159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ab9357783b419a83e192123b581543', 'request_id': 'req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.478177 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ab9357783b419a83e192123b581543', 'request_id': 'req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.491597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] The token cache is being invalidate because project 40ab9357783b419a83e192123b581543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:54.491884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.492584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb1964c5-fe05-41dd-8ae5-c6c66a58d47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:54.494191 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1378/2750] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:54 2022] DELETE /identity/v3/projects/40ab9357783b419a83e192123b581543 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:54.500900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.501299 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.501647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.561679 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:54.562905 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:54.626969 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1373/2751] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:54.638993 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.647813 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.648160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.668754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:54.670046 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:54.678501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1379/2752] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:54.682956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.689398 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:54.693567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f74d67-83f3-403f-bd42-562d77f3f8e6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.699786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.701223 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.701942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.702729 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.703809 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:54.704367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:54.706756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:54.722246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1374/2753] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 14:08:54.726838 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.737007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7a93a-a27f-4ea7-bdd7-31bc0eef239b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.741985 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.742665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.742949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.743226 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.743941 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:54.745917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:54.778014 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1380/2754] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:54 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:54.784127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.794715 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865fc80-c942-467b-9c94-e33680da368b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.856971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.857652 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.857940 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.858216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.858914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:54.861164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:54.884620 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1375/2755] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:54 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:54.889796 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.907275 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82d84-39b9-42d6-9e8b-b98d616b747e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.914808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.915788 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.916238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.917337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.917969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:54.920079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:54.937661 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1381/2756] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:54.942111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.953774 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15578b9-58e6-4604-b923-9bf40f9077d4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.959207 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9422c0-3cf6-4375-acb2-20401b99faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.960044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9422c0-3cf6-4375-acb2-20401b99faf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.960370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9422c0-3cf6-4375-acb2-20401b99faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.960758 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9422c0-3cf6-4375-acb2-20401b99faf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:54.961526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9422c0-3cf6-4375-acb2-20401b99faf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:54.963385 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9422c0-3cf6-4375-acb2-20401b99faf0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:54.975186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1376/2757] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:54.979782 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:54.991947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007e6a-760d-4c5e-b8a5-bae9cf410a2b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:54.997820 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:54.998724 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:54.999001 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:54.999277 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] PATH_INFO: `/v3/projects/4e89d1426fa746dd81b4d8fab49a4317/users/32fe584112614c2fbc36d9d035b7181d/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.016503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e89d1426fa746dd81b4d8fab49a4317, user_id=32fe584112614c2fbc36d9d035b7181d, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:55.019194 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:55.033535 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1382/2758] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:54 2022] PUT /identity/v3/projects/4e89d1426fa746dd81b4d8fab49a4317/users/32fe584112614c2fbc36d9d035b7181d/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:55.044224 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:55.044544 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:55.044850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.074292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] MFA Rules not processed for user `32fe584112614c2fbc36d9d035b7181d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:55.076198 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:55.166567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1377/2759] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:55.201893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:55.203673 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:55.267952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f31cf-ea36-4b58-b3f0-df4984632ad3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:55.272124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052978b-2f79-4637-8034-6a6eac7d7ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:55.272818 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052978b-2f79-4637-8034-6a6eac7d7ada service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:55.272971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052978b-2f79-4637-8034-6a6eac7d7ada service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:55.273206 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052978b-2f79-4637-8034-6a6eac7d7ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.282390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052978b-2f79-4637-8034-6a6eac7d7ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:55.283825 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052978b-2f79-4637-8034-6a6eac7d7ada service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:55.293129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1383/2760] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:08:55.311986 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:55.377815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c123f-64f9-4bc9-ab43-5a912bde3386 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:55.382797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:55.383683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:55.383968 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:55.384244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] PATH_INFO: `/v3/users/32fe584112614c2fbc36d9d035b7181d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.385515 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fe584112614c2fbc36d9d035b7181d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:55.387454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:55.389209 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:55.446151 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fe584112614c2fbc36d9d035b7181d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:55.459148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fe584112614c2fbc36d9d035b7181d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:55.466098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fe584112614c2fbc36d9d035b7181d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:55.475557 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fe584112614c2fbc36d9d035b7181d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:55.477069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fe584112614c2fbc36d9d035b7181d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:55.485885 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1378/2761] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:08:55 2022] DELETE /identity/v3/users/32fe584112614c2fbc36d9d035b7181d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:55.490514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:55.491083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:55.491083 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.509338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:55.510892 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:55.520947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:55.573909 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1384/2762] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:55.583772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:55.585241 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:55.647111 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e8ffa-5739-4e1f-b52b-584b8aa97d8f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:55.650823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe6ac6-1c99-4555-be4e-337c6e0c8c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:55.651479 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe6ac6-1c99-4555-be4e-337c6e0c8c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:55.651761 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe6ac6-1c99-4555-be4e-337c6e0c8c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:55.652033 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe6ac6-1c99-4555-be4e-337c6e0c8c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.662273 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe6ac6-1c99-4555-be4e-337c6e0c8c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:55.663701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe6ac6-1c99-4555-be4e-337c6e0c8c37 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:55.673864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1379/2763] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:08:55.961509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:55.963107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:55.978416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a986eaa-0c51-4ab0-885e-c8ebac8766d7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:55.984904 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:55.986062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:55.986556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:55.986645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] PATH_INFO: `/v3/projects/4e89d1426fa746dd81b4d8fab49a4317` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:55.987963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e89d1426fa746dd81b4d8fab49a4317)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:55.989975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.045933 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e89d1426fa746dd81b4d8fab49a4317', 'request_id': 'req-cbefd39b-93b5-4845-80ca-28fb189c2611'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:56.052747 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e89d1426fa746dd81b4d8fab49a4317', 'request_id': 'req-cbefd39b-93b5-4845-80ca-28fb189c2611'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:56.061404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e89d1426fa746dd81b4d8fab49a4317', 'request_id': 'req-cbefd39b-93b5-4845-80ca-28fb189c2611'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:56.080695 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] The token cache is being invalidate because project 4e89d1426fa746dd81b4d8fab49a4317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:08:56.081197 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:56.083123 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbefd39b-93b5-4845-80ca-28fb189c2611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:08:56.085951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1385/2764] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:08:55 2022] DELETE /identity/v3/projects/4e89d1426fa746dd81b4d8fab49a4317 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:56.098084 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.098979 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.099147 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.121946 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:56.123169 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:56.191514 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1380/2765] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:08:56.199271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.199566 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.199850 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.219784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:56.221049 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:56.230929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1386/2766] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:56.235346 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.246043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce5791-b860-46f6-acb2-8bfc4f6589d9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.250852 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.251529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.251815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.252117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.252894 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:08:56.253205 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.254818 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.270520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1381/2767] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:08:56.274915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.285166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917df90-1d9c-4832-82eb-fc8dbc096045 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.289905 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.290569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.290851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.291124 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.291842 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.293770 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.321664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1387/2768] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:08:56 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:56.326627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.337172 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978be370-3b4c-4cae-95f1-4bf1fbbc3591 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.342644 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:56.396780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.397455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.397731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.397999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.398756 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.400638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.423541 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1382/2769] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:08:56 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:08:56.431560 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.446365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c239048-c445-42ab-b0d5-6af9cac3360f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.451296 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.451972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.452257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.452564 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.453308 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.454918 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.468238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1388/2770] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:56.472597 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.483396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c541-8ae0-4aa0-87ae-eb065d4a060e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.488911 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee094e-f004-4838-b651-a09bf1085dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.489595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee094e-f004-4838-b651-a09bf1085dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.489882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee094e-f004-4838-b651-a09bf1085dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.490161 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee094e-f004-4838-b651-a09bf1085dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.490906 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee094e-f004-4838-b651-a09bf1085dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.492539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee094e-f004-4838-b651-a09bf1085dc4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.503854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1383/2771] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:08:56.508030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.519230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85b0a-39e1-44c2-8145-4d9b0c75540b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.524893 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.525794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.526082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.526365 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] PATH_INFO: `/v3/projects/ddbd6e1cb0814523b4e528c14d06cdad/users/1e956289b0e64cc892d453f427845d71/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.542989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbd6e1cb0814523b4e528c14d06cdad, user_id=1e956289b0e64cc892d453f427845d71, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.545697 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.559222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1389/2772] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:08:56 2022] PUT /identity/v3/projects/ddbd6e1cb0814523b4e528c14d06cdad/users/1e956289b0e64cc892d453f427845d71/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:08:56.565028 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83c3a-79f9-422c-822c-d196d333ef53 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.565440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83c3a-79f9-422c-822c-d196d333ef53 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.565824 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83c3a-79f9-422c-822c-d196d333ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.585627 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1dd83c3a-79f9-422c-822c-d196d333ef53 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:08:56.586775 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1dd83c3a-79f9-422c-822c-d196d333ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:56.647577 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1384/2773] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:08:56.657482 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:08:56.659217 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:08:56.699504 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:08:56.722744 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694ff3a-8011-44eb-a4fc-28db9497adc3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:08:56.726463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:08:56.727120 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:08:56.727410 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:08:56.727688 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:08:56.737879 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:08:56.739370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:08:56.750817 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1390/2774] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:09:01.155226 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.155862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.155862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.188100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] MFA Rules not processed for user `1e956289b0e64cc892d453f427845d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:09:01.189438 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:01.271822 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1385/2775] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:09:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:09:01.276003 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.286434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e3f7-291d-4102-a1b1-2a9e4c770def service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.293017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:01.293791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.294508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.294851 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.295551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:01.297447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:01.329552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1391/2776] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:09:01 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:09:01.333763 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.346074 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59893010-bffb-45d7-a289-80cb7d172ed3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.361179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:09:01.409646 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:01.410326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.410635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.410914 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.411618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:01.413645 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:01.436797 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1386/2777] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:09:01 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:09:01.440769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.452190 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dff3d-20ca-4549-84ce-1cdaa27d1c24 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.457603 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:01.458300 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.458589 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.458865 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.459613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:01.461314 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:01.485315 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1392/2778] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:09:01.487043 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.506671 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1be423-68c6-4a49-890d-7d891ba4150b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.522047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:01.523883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.523883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.523883 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] PATH_INFO: `/v3/projects/926204d660974b039dfb2d21b87d8d7d/users/a003de4e882049479c9aa10b36495ee9/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.548337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926204d660974b039dfb2d21b87d8d7d, user_id=a003de4e882049479c9aa10b36495ee9, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:01.551067 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:01.575648 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1387/2779] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:09:01 2022] PUT /identity/v3/projects/926204d660974b039dfb2d21b87d8d7d/users/a003de4e882049479c9aa10b36495ee9/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:09:01.580200 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.596481 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb17c1-3e75-4982-8bef-be300ae0c5f9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.678181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:01.679137 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.679874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.679874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.681352 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:01.683220 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:01.695608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1393/2780] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:09:01.699762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.710917 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7ff2-daf4-45f1-a8c7-7dbb1ff72de2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.716301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244f392-ff3b-4d36-ba36-bfee02c14a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:01.717375 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244f392-ff3b-4d36-ba36-bfee02c14a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:01.717725 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244f392-ff3b-4d36-ba36-bfee02c14a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:01.718066 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244f392-ff3b-4d36-ba36-bfee02c14a99 admin admin] PATH_INFO: `/v3/projects/926204d660974b039dfb2d21b87d8d7d/users/a003de4e882049479c9aa10b36495ee9/roles/10895c39249448e4ac3616747cafa76b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:01.720343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244f392-ff3b-4d36-ba36-bfee02c14a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926204d660974b039dfb2d21b87d8d7d, user_id=a003de4e882049479c9aa10b36495ee9, role_id=10895c39249448e4ac3616747cafa76b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:01.723376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244f392-ff3b-4d36-ba36-bfee02c14a99 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:01.742337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1388/2781] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:09:01 2022] PUT /identity/v3/projects/926204d660974b039dfb2d21b87d8d7d/users/a003de4e882049479c9aa10b36495ee9/roles/10895c39249448e4ac3616747cafa76b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:09:01.828597 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:01.830599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:01.946265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073003d-55e9-4787-96e2-dbaf4915308c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:01.976072 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:09:02.042117 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:02.043021 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:02.043502 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:02.044176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:02.047487 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:02.070295 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:02.073110 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:02.085769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1394/2782] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:09:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:09:06.728144 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8ccf3-4460-4002-8876-c33b4ebeacff None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:06.728759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8ccf3-4460-4002-8876-c33b4ebeacff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:06.728870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8ccf3-4460-4002-8876-c33b4ebeacff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:06.768050 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4dc8ccf3-4460-4002-8876-c33b4ebeacff None None] MFA Rules not processed for user `a003de4e882049479c9aa10b36495ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:09:06.769495 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4dc8ccf3-4460-4002-8876-c33b4ebeacff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:06.860971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1389/2783] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:09:06.871795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:06.873289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:06.994612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2682065-c90f-4ccd-83b4-12d0162aa732 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:07.001524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3adbe-a957-4200-b3a8-97614f67aa47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:07.002301 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3adbe-a957-4200-b3a8-97614f67aa47 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:07.002669 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3adbe-a957-4200-b3a8-97614f67aa47 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:07.002949 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3adbe-a957-4200-b3a8-97614f67aa47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:07.074145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3adbe-a957-4200-b3a8-97614f67aa47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:07.075740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3adbe-a957-4200-b3a8-97614f67aa47 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:07.086294 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1395/2784] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:09:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:09:16.849373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:09:17.081404 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:09:23.712629 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:23.713240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:23.713240 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:23.730713 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] MFA Rules not processed for user `1e956289b0e64cc892d453f427845d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:09:23.731976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:23.739804 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1390/2785] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:09:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:09:31.593233 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:31.593810 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:31.593923 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:31.611574 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] MFA Rules not processed for user `4a3a211465a0478994d011f89eaf972f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:09:31.612982 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:31.669225 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1396/2786] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:09:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:09:33.725476 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:09:41.339056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:09:41.340759 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:09:41.429882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faafe03-2bc4-4441-80cb-ca42cdcceb0f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:09:41.433530 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:09:41.434182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:09:41.434465 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:09:41.434742 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:09:41.445164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:09:41.446618 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:09:41.457539 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1391/2787] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:09:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:09:41.662532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=25 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:09:51.452900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=35 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:10:24.761662 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:10:24.783660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e78d-5fae-4f25-b854-7e05c056f0a2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:10:24.789919 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:10:24.791078 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:10:24.791972 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:10:24.792624 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:10:24.795924 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:10:24.816265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:10:24.816265 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61af4e-b27c-48a0-8bab-5661748732a7 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:10:24.824953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1397/2788] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:10:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:10:24.925504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:10:24.937186 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29328fb-86cc-41a6-a1dd-485b061c1cf7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:10:24.941959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:10:24.942832 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:10:24.943213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:10:24.943585 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:10:24.945913 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:10:24.965583 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:10:24.967370 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:10:24.979807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1392/2789] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:10:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:10:34.821370 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:10:34.974599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:10:37.698533 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:10:37.699099 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:10:37.699288 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:10:37.719269 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] MFA Rules not processed for user `4a3a211465a0478994d011f89eaf972f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:10:37.720645 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:10:37.727836 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1398/2790] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:10:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:10:47.713027 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:11:30.515318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:30.517524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:30.610469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f8436-78fb-4d01-a1ed-60b584e0ef40 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:30.615268 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:30.616675 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:30.616984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:30.617286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:30.619246 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:30.634801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:30.636236 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961ca54-c34b-4831-86c0-69d565c9c355 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:30.648039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1393/2791] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:11:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:11:38.545931 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:38.547462 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:38.559468 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde97c-5adf-4d4d-ba71-882cc592097d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:38.562975 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:38.563616 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:38.563896 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:38.564170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:38.566165 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:38.583493 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:38.586085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:38.600348 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1399/2792] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:11:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:11:40.642520 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=35 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:11:46.389001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fadee8-a8d6-415b-be80-310583372233 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:46.389638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fadee8-a8d6-415b-be80-310583372233 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:46.389638 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fadee8-a8d6-415b-be80-310583372233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:46.409450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-88fadee8-a8d6-415b-be80-310583372233 None None] MFA Rules not processed for user `1e956289b0e64cc892d453f427845d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:11:46.410699 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-88fadee8-a8d6-415b-be80-310583372233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:46.418636 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1394/2793] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:11:47.317236 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:47.319716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:47.339171 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86d707-6c9f-462a-aa18-c16c5700b619 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:47.344903 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:47.345629 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:47.345980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:47.346318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:47.348167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:47.363477 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:47.366306 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:47.387252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1400/2794] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:11:49.328528 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fadee8-a8d6-415b-be80-310583372233 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:49.339216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fadee8-a8d6-415b-be80-310583372233 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:49.342918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:49.343584 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:49.343871 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:49.344150 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:49.346016 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:49.357239 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:49.358702 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:49.369582 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1395/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:11:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:11:50.034526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:50.036839 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:50.056643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579831c-e53b-41fa-9ec2-41e00ee23041 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:50.070131 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:50.071223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:50.071730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:50.072239 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:50.074441 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:50.090271 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:50.092235 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:50.105135 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1401/2796] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:11:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:11:53.874460 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:53.876294 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:53.947260 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c5af3-ea6f-450a-9690-4dd9d19cac52 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:53.951838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:53.952749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:53.953039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:53.953322 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] PATH_INFO: `/v3/users/4a3a211465a0478994d011f89eaf972f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:53.954591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a3a211465a0478994d011f89eaf972f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:53.956602 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.004469 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a3a211465a0478994d011f89eaf972f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.016815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a3a211465a0478994d011f89eaf972f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.023833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a3a211465a0478994d011f89eaf972f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.030770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a3a211465a0478994d011f89eaf972f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.031384 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-871a44a9-22d0-424c-bc53-fc44341fa83b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a3a211465a0478994d011f89eaf972f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.037517 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1396/2797] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:11:53 2022] DELETE /identity/v3/users/4a3a211465a0478994d011f89eaf972f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:11:54.222614 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.235696 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bccf21-d5c5-4792-a1f1-2abc5fd1c462 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.297551 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.298432 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.298718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.298997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] PATH_INFO: `/v3/projects/26daacf03c844ff8a7546dfe3b36d8bb` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.300605 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26daacf03c844ff8a7546dfe3b36d8bb)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.303095 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.354229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26daacf03c844ff8a7546dfe3b36d8bb', 'request_id': 'req-739742b3-590a-4562-bad3-1f8e58cd6138'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.360776 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26daacf03c844ff8a7546dfe3b36d8bb', 'request_id': 'req-739742b3-590a-4562-bad3-1f8e58cd6138'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.369769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26daacf03c844ff8a7546dfe3b36d8bb', 'request_id': 'req-739742b3-590a-4562-bad3-1f8e58cd6138'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.382252 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] The token cache is being invalidate because project 26daacf03c844ff8a7546dfe3b36d8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:11:54.382538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.383194 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-739742b3-590a-4562-bad3-1f8e58cd6138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:11:54.384794 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1402/2798] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:11:54 2022] DELETE /identity/v3/projects/26daacf03c844ff8a7546dfe3b36d8bb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:11:54.390920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.391171 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.391454 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.410307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:11:54.411529 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:54.467678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:11:54.486845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1397/2799] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:11:54.497238 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.497671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.498158 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.523591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:11:54.524916 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:54.533505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1403/2800] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:11:54.538590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.548484 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cfde2-bd8a-4b7e-9aed-00fe5eebd43b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.553858 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.554566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.554850 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.555167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.555907 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:11:54.556216 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.557862 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.570468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1398/2801] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:11:54.574964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.584297 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd894f-cc88-4c9d-a273-972046373a66 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.588968 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.589626 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.589906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.590186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.590901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.592861 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.632709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1404/2802] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:11:54 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:11:54.635989 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.646565 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914fe9-977a-4514-81c2-7cff085496ad admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.705386 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.706038 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.706321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.706595 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.707283 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.709865 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.733623 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1399/2803] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:11:54 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:11:54.734771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.745474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc721da-9118-48f6-bf6f-990694f050e5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.751473 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.752149 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.752460 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.753336 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.753486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.755122 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.768853 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1405/2804] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:11:54.770042 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.781149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2378028-b105-46c5-baff-3941582dca01 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.786045 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f30cf-fee6-436b-89d2-9694612d04a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.787292 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f30cf-fee6-436b-89d2-9694612d04a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.787635 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f30cf-fee6-436b-89d2-9694612d04a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.787845 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f30cf-fee6-436b-89d2-9694612d04a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.788621 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f30cf-fee6-436b-89d2-9694612d04a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.790320 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f30cf-fee6-436b-89d2-9694612d04a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.800749 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1400/2805] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:11:54.805096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.810290 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:11:54.816785 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535db17-7c81-4815-be36-2a926ae609e1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:54.822172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:54.823058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.823340 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.823612 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] PATH_INFO: `/v3/projects/cf0ff2ac8a1d48e0af0fcc3b56155ede/users/bd0ea17617ec4a2f83da81a01ee2ca0a/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.838653 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0ff2ac8a1d48e0af0fcc3b56155ede, user_id=bd0ea17617ec4a2f83da81a01ee2ca0a, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:54.841298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:54.854577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1406/2806] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:11:54 2022] PUT /identity/v3/projects/cf0ff2ac8a1d48e0af0fcc3b56155ede/users/bd0ea17617ec4a2f83da81a01ee2ca0a/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:11:54.865722 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:54.866013 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:54.866297 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:54.892857 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] MFA Rules not processed for user `bd0ea17617ec4a2f83da81a01ee2ca0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:11:54.894063 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:54.957275 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1401/2807] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:11:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:11:54.968660 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:11:54.970337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:11:55.028701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3439bb4-b130-49f2-bc5d-3968186cc254 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:11:55.032349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:11:55.033006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:11:55.033289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:11:55.033565 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:11:55.041934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:11:55.043371 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:11:55.052534 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1407/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:11:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:12:02.229445 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:02.231126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:02.292566 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:02.314462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72329c3-0548-443a-8d20-a8aee89c96e7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:02.320823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:02.321488 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:02.321770 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:02.322046 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:02.323870 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:02.336753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:02.338353 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406cf28-22bc-4cc1-ab90-0843bb2dc7e6 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:02.349932 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1402/2809] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:12:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:12:05.048176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:12.344954 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=9 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:15.072443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:15.163571 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a72b-48f8-4069-99bd-1fc377862baf service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:15.166476 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:15.167543 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:15.167890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:15.168374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] PATH_INFO: `/v3/users/bd0ea17617ec4a2f83da81a01ee2ca0a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:15.169822 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0ea17617ec4a2f83da81a01ee2ca0a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:15.172103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:15.281307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0ea17617ec4a2f83da81a01ee2ca0a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.308183 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0ea17617ec4a2f83da81a01ee2ca0a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.324771 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0ea17617ec4a2f83da81a01ee2ca0a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.325562 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0ea17617ec4a2f83da81a01ee2ca0a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.326259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0ea17617ec4a2f83da81a01ee2ca0a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.344934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1408/2810] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:12:15 2022] DELETE /identity/v3/users/bd0ea17617ec4a2f83da81a01ee2ca0a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:12:15.350999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:15.350999 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:15.351833 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:15.395610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:15.397956 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:15.493846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1403/2811] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:12:15.500447 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:15.502423 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:15.635446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8094fe-cc36-4daa-99c7-590c6c2a8ffe admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:15.642667 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:15.643827 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:15.644343 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:15.646957 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:15.669334 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:15.672140 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:15.691519 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1409/2812] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:12:15.709524 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:15.711794 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:15.725400 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7524c-e550-4bd7-8199-3a3e0a2875fd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:15.731276 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:15.732436 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:15.732882 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:15.733321 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] PATH_INFO: `/v3/users/f72d7744f4c4493a8e2a5f87e9a89d5c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:15.734640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72d7744f4c4493a8e2a5f87e9a89d5c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:15.736640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:15.806962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72d7744f4c4493a8e2a5f87e9a89d5c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.823338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72d7744f4c4493a8e2a5f87e9a89d5c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.830962 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72d7744f4c4493a8e2a5f87e9a89d5c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.843081 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72d7744f4c4493a8e2a5f87e9a89d5c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.844414 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72d7744f4c4493a8e2a5f87e9a89d5c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:15.852734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1404/2813] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:12:15 2022] DELETE /identity/v3/users/f72d7744f4c4493a8e2a5f87e9a89d5c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:12:16.057592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.060150 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.072855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.074639 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.111469 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2df5ef-19d6-43f7-870c-a17caf1ffb76 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.146229 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:16.159697 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086c69d-24b7-451d-a70f-6f0f343b809f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.165881 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.166864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.167324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.167806 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] PATH_INFO: `/v3/projects/e608cc75a5ed4f05b462c84c29a31788` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.169149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e608cc75a5ed4f05b462c84c29a31788)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.171926 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.192642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:16.196860 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.197855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.198208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.198552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] PATH_INFO: `/v3/projects/cf0ff2ac8a1d48e0af0fcc3b56155ede` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.199953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0ff2ac8a1d48e0af0fcc3b56155ede)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.202221 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.235318 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e608cc75a5ed4f05b462c84c29a31788', 'request_id': 'req-d73a38b1-31f8-4618-8795-f0ebd47db3b0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.244563 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e608cc75a5ed4f05b462c84c29a31788', 'request_id': 'req-d73a38b1-31f8-4618-8795-f0ebd47db3b0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.262367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e608cc75a5ed4f05b462c84c29a31788', 'request_id': 'req-d73a38b1-31f8-4618-8795-f0ebd47db3b0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.272332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0ff2ac8a1d48e0af0fcc3b56155ede', 'request_id': 'req-22ef6fa3-a717-4416-b7e8-ad12257262a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.274588 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] The token cache is being invalidate because project e608cc75a5ed4f05b462c84c29a31788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:12:16.274942 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.275661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.278737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1405/2814] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:12:16 2022] DELETE /identity/v3/projects/e608cc75a5ed4f05b462c84c29a31788 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:12:16.280274 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0ff2ac8a1d48e0af0fcc3b56155ede', 'request_id': 'req-22ef6fa3-a717-4416-b7e8-ad12257262a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.282949 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.284764 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.290664 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0ff2ac8a1d48e0af0fcc3b56155ede', 'request_id': 'req-22ef6fa3-a717-4416-b7e8-ad12257262a0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.306538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] The token cache is being invalidate because project cf0ff2ac8a1d48e0af0fcc3b56155ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:12:16.306829 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.307542 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22ef6fa3-a717-4416-b7e8-ad12257262a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.309576 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1410/2815] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:12:16 2022] DELETE /identity/v3/projects/cf0ff2ac8a1d48e0af0fcc3b56155ede => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:12:16.315474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa227-2234-49db-8d99-c25fa0e65a5d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.315884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa227-2234-49db-8d99-c25fa0e65a5d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.316096 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa227-2234-49db-8d99-c25fa0e65a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.336781 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cb9aa227-2234-49db-8d99-c25fa0e65a5d None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:16.338017 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cb9aa227-2234-49db-8d99-c25fa0e65a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.365696 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a38b1-31f8-4618-8795-f0ebd47db3b0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.418285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1411/2816] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:12:16.430328 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.430665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.430665 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.431030 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.431199 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.431478 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.431757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] PATH_INFO: `/v3/users/be679ac36101449d8bad2fef00783737` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.433055 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be679ac36101449d8bad2fef00783737)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.434995 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.450964 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:16.452996 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.461338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1412/2817] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:12:16.466833 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.481901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13f7e1-cd82-47cb-9e47-2d1661d60d8b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.488438 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.489337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.489736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.490159 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.491145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:12:16.491671 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.493306 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be679ac36101449d8bad2fef00783737'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.493864 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.498286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:16.508989 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1413/2818] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:12:16.510678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be679ac36101449d8bad2fef00783737'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.513917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.522089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be679ac36101449d8bad2fef00783737'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.524913 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b363b-3ee7-4038-9cfa-7f5db0b31e46 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.530102 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.530784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.531079 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.531359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.531681 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be679ac36101449d8bad2fef00783737'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.532086 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.532670 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be679ac36101449d8bad2fef00783737'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.534073 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.541493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1406/2819] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:12:16 2022] DELETE /identity/v3/users/be679ac36101449d8bad2fef00783737 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:12:16.568242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1414/2820] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:12:16 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:12:16.569674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.571096 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.667811 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621412e-765f-463a-99a2-e732772cbfa9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.750359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.750662 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.751437 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.751785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.752116 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.753253 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.754606 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.755388 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.772335 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6e8c6-b50f-4630-af17-e1266381ce5a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.780304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.781384 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.781775 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.782052 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] PATH_INFO: `/v3/projects/45cc3efc2301469d897af308ce66fde6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.783456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45cc3efc2301469d897af308ce66fde6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.785934 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.788490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1407/2821] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:12:16 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:12:16.792762 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.794074 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:16.804099 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a148d5-b7b2-4399-93b2-50f7ceafdb58 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.809030 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.809731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.810015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.810298 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.811036 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.812733 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.824753 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1408/2822] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:12:16.829126 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.839338 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5b75a-4de5-47a7-b718-f7787b5e6828 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.842005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45cc3efc2301469d897af308ce66fde6', 'request_id': 'req-0f31fa69-8d6e-4979-887a-c4d9f873bee8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.848890 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45cc3efc2301469d897af308ce66fde6', 'request_id': 'req-0f31fa69-8d6e-4979-887a-c4d9f873bee8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.857408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45cc3efc2301469d897af308ce66fde6', 'request_id': 'req-0f31fa69-8d6e-4979-887a-c4d9f873bee8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.868986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] The token cache is being invalidate because project 45cc3efc2301469d897af308ce66fde6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:12:16.869284 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.869953 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:12:16.871541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1415/2823] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:12:16 2022] DELETE /identity/v3/projects/45cc3efc2301469d897af308ce66fde6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:12:16.901058 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:16.909630 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ee6c1-cb60-4c8f-97b4-9ce5acf328b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.911201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee6c1-cb60-4c8f-97b4-9ce5acf328b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.911201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee6c1-cb60-4c8f-97b4-9ce5acf328b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.911201 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee6c1-cb60-4c8f-97b4-9ce5acf328b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.912650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ee6c1-cb60-4c8f-97b4-9ce5acf328b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.913984 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867ee6c1-cb60-4c8f-97b4-9ce5acf328b5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:16.932432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1409/2824] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:12:16.937012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:16.939578 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:16.957601 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31fa69-8d6e-4979-887a-c4d9f873bee8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:16.963278 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:16.964209 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:16.964523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:16.964808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] PATH_INFO: `/v3/projects/18d5cd47ac9f408385f194b94f17049b/users/9c0337148d584c14a3dc0fc7ccff766b/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:16.982405 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d5cd47ac9f408385f194b94f17049b, user_id=9c0337148d584c14a3dc0fc7ccff766b, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:16.985401 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:17.004129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1416/2825] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:12:16 2022] PUT /identity/v3/projects/18d5cd47ac9f408385f194b94f17049b/users/9c0337148d584c14a3dc0fc7ccff766b/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:12:17.009854 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e90b9-f286-457a-bf7a-120950bf1a72 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:17.010230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e90b9-f286-457a-bf7a-120950bf1a72 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:17.010611 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e90b9-f286-457a-bf7a-120950bf1a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:17.031837 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cf0e90b9-f286-457a-bf7a-120950bf1a72 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:17.033140 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cf0e90b9-f286-457a-bf7a-120950bf1a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:17.119985 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1410/2826] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:12:17.133434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:17.135454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:17.228717 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05475d-6b9f-4f37-9aca-68be3000285d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:17.233509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:17.234585 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:17.235011 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:17.235501 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:17.252240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:17.255133 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:17.274328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1417/2827] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:12:18.027331 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701ebab-b687-49e8-affe-9276938a1b5d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:18.027912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701ebab-b687-49e8-affe-9276938a1b5d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:18.027912 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701ebab-b687-49e8-affe-9276938a1b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:18.057432 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0701ebab-b687-49e8-affe-9276938a1b5d None None] MFA Rules not processed for user `9c0337148d584c14a3dc0fc7ccff766b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:18.058678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0701ebab-b687-49e8-affe-9276938a1b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:18.139957 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1411/2828] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:12:18.155471 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:18.156955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:18.235668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf09465-fa6c-4f87-9237-cb7c48361048 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:18.240435 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15159282-5650-48ab-8dda-7f5ac8badca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:18.241390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15159282-5650-48ab-8dda-7f5ac8badca4 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:18.241716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15159282-5650-48ab-8dda-7f5ac8badca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:18.242176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15159282-5650-48ab-8dda-7f5ac8badca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:18.258291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15159282-5650-48ab-8dda-7f5ac8badca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:18.261193 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15159282-5650-48ab-8dda-7f5ac8badca4 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:18.282848 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1418/2829] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:12:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:12:19.684637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea502a-1585-49d9-81eb-62919932bf3f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:19.685390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea502a-1585-49d9-81eb-62919932bf3f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:19.685390 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea502a-1585-49d9-81eb-62919932bf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:19.705965 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-02ea502a-1585-49d9-81eb-62919932bf3f None None] MFA Rules not processed for user `9c0337148d584c14a3dc0fc7ccff766b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:19.707182 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-02ea502a-1585-49d9-81eb-62919932bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:19.714654 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1412/2830] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:12:28.276505 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:29.700310 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=66 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:33.904377 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:33.904995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:33.904995 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:33.925525 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] MFA Rules not processed for user `9c0337148d584c14a3dc0fc7ccff766b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:33.926797 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:33.934104 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1419/2831] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:12:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:12:43.920153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:12:52.680634 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398b8b1-a316-4f8b-9414-e58eb77df6c8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:52.681178 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398b8b1-a316-4f8b-9414-e58eb77df6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:52.681301 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398b8b1-a316-4f8b-9414-e58eb77df6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:52.749366 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1398b8b1-a316-4f8b-9414-e58eb77df6c8 None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:12:52.750664 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1398b8b1-a316-4f8b-9414-e58eb77df6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:52.825148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1413/2832] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jul 15 14:12:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:12:52.882376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:12:52.884188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:12:53.129703 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb25d6-10e2-4ec8-8f18-8fe2bb792469 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:12:53.135385 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:12:53.136172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:12:53.136450 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:12:53.136756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:12:53.146240 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:12:53.147857 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:12:53.157791 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1420/2833] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:12:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:13:02.821799 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=44 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:03.153668 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=41 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:08.601569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:08.602961 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:08.603441 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:08.630462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] MFA Rules not processed for user `9459ea72747e4ad4a839847d8232bb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:08.632613 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:08.642426 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1414/2834] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jul 15 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:08.658060 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:08.670906 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71014be2-4a94-4f5e-bf25-4d8757ea9a23 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:08.676309 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3eacd1356f448cca079f57f683f37bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:08.677915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:08.678192 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:08.678463 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:08.688015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:08.689648 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:08.700429 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1421/2835] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jul 15 14:13:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 14:13:10.609180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:10.610743 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:10.666486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:10.684307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131c782-4b06-4af8-9ada-49e8237bea48 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:10.692216 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:10.693765 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:10.694119 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:10.694395 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] PATH_INFO: `/v3/users/9c0337148d584c14a3dc0fc7ccff766b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:10.695615 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0337148d584c14a3dc0fc7ccff766b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:10.697547 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:10.697976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd1ece-bf92-4494-a32c-974c7e34276e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:10.703778 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:10.704486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:10.704779 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:10.705057 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:10.706910 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:10.719173 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:10.720886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:10.732895 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1415/2836] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:13:10.776442 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0337148d584c14a3dc0fc7ccff766b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:10.789886 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0337148d584c14a3dc0fc7ccff766b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:10.799641 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0337148d584c14a3dc0fc7ccff766b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:10.800702 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0337148d584c14a3dc0fc7ccff766b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:10.801319 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-aa835e2c-3af2-4652-bd4d-70be4d7ffd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0337148d584c14a3dc0fc7ccff766b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:10.815208 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1422/2837] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:13:10 2022] DELETE /identity/v3/users/9c0337148d584c14a3dc0fc7ccff766b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:11.004668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.016164 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ec803-b04b-402b-a0e0-e1997c4bce58 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.081997 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.082859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.083142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.083615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] PATH_INFO: `/v3/projects/18d5cd47ac9f408385f194b94f17049b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.084855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d5cd47ac9f408385f194b94f17049b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.086886 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.134416 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d5cd47ac9f408385f194b94f17049b', 'request_id': 'req-24fd0a1e-818d-490b-b244-3c5df5b2f933'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:11.140485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d5cd47ac9f408385f194b94f17049b', 'request_id': 'req-24fd0a1e-818d-490b-b244-3c5df5b2f933'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:11.149383 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d5cd47ac9f408385f194b94f17049b', 'request_id': 'req-24fd0a1e-818d-490b-b244-3c5df5b2f933'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:11.160569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] The token cache is being invalidate because project 18d5cd47ac9f408385f194b94f17049b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:13:11.160859 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:11.161575 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24fd0a1e-818d-490b-b244-3c5df5b2f933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:11.163642 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1416/2838] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:13:11 2022] DELETE /identity/v3/projects/18d5cd47ac9f408385f194b94f17049b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:11.168591 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.168956 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.169367 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.187456 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:11.188684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:11.246639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1423/2839] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:13:11.254423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.254711 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.254990 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.273674 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:11.274877 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:11.282817 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1417/2840] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:11.287792 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.297737 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a916ff9-37e4-48d2-ae87-bc9dd319545b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.302316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.302967 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.303257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.303526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.304254 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:13:11.304599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.306182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.317680 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1424/2841] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:13:11.322089 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.331285 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3de99-2404-4a73-a2af-6e70530a7b07 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.336473 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.337118 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.337397 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.337669 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.338363 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.340254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.343444 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:11.369358 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1418/2842] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:13:11 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:13:11.373541 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.383526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83876622-3bb9-4580-b4e1-12ce525bcfc1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.428699 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:11.443740 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.444443 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.444730 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.445007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.445831 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.447622 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.470816 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1425/2843] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:13:11 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:13:11.475676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.486341 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c8173-ec56-4df8-be69-3067f7f1a955 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.491405 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.492073 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.492354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.492661 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.493396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.495001 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.506145 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1419/2844] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:13:11.510324 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.520963 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d96a05-bfc4-40ac-b46f-42dd0bbdcbcf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.526716 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25deb059-f8b2-4d57-a66e-0ad360d48d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.527390 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25deb059-f8b2-4d57-a66e-0ad360d48d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.527705 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25deb059-f8b2-4d57-a66e-0ad360d48d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.527986 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25deb059-f8b2-4d57-a66e-0ad360d48d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.528777 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25deb059-f8b2-4d57-a66e-0ad360d48d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.530425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25deb059-f8b2-4d57-a66e-0ad360d48d41 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.541098 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1426/2845] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:13:11.545376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.555572 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b31ffc-892d-499e-bb68-8966078640b2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.560259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.561210 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.561493 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.561772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] PATH_INFO: `/v3/projects/85c01920348f4ebeb98c5ea2f3ddde93/users/604b7778fa9249169f7bcea92a21f701/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.576757 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c01920348f4ebeb98c5ea2f3ddde93, user_id=604b7778fa9249169f7bcea92a21f701, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.579431 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.593142 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1420/2846] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:13:11 2022] PUT /identity/v3/projects/85c01920348f4ebeb98c5ea2f3ddde93/users/604b7778fa9249169f7bcea92a21f701/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:11.603214 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.603514 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.603798 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.629962 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] MFA Rules not processed for user `604b7778fa9249169f7bcea92a21f701`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:11.631769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:11.697157 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1427/2847] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:11.708316 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:11.710686 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:11.766526 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be67a5-d84c-472e-b526-239e514388bb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:11.770044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b24cb-6e4f-4504-aa49-8ad75232aae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:11.770683 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b24cb-6e4f-4504-aa49-8ad75232aae8 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:11.771510 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b24cb-6e4f-4504-aa49-8ad75232aae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:11.771785 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b24cb-6e4f-4504-aa49-8ad75232aae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:11.780286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b24cb-6e4f-4504-aa49-8ad75232aae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:11.781783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b24cb-6e4f-4504-aa49-8ad75232aae8 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:11.790959 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1421/2848] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:13:14.320486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:14.384376 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870541e7-0c58-41f2-aead-1d8ecbc1ed4b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:14.388977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:14.389999 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:14.390379 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:14.390763 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] PATH_INFO: `/v3/users/604b7778fa9249169f7bcea92a21f701` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:14.392145 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=604b7778fa9249169f7bcea92a21f701)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:14.394328 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:14.445567 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604b7778fa9249169f7bcea92a21f701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:14.452772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:14.459888 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604b7778fa9249169f7bcea92a21f701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:14.467544 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604b7778fa9249169f7bcea92a21f701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:14.468182 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604b7778fa9249169f7bcea92a21f701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:14.468784 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604b7778fa9249169f7bcea92a21f701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:14.481980 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1428/2849] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:13:14 2022] DELETE /identity/v3/users/604b7778fa9249169f7bcea92a21f701 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:14.487608 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:14.488430 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:14.488687 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:14.510325 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:14.512091 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:14.575213 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1422/2850] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:14.585604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:14.587185 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:14.649955 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31c07a-76df-4a53-976a-e048b97c10fd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:14.653976 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149520-eef2-47a6-a360-009e7b665eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:14.654633 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149520-eef2-47a6-a360-009e7b665eab service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:14.654948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149520-eef2-47a6-a360-009e7b665eab service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:14.655210 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149520-eef2-47a6-a360-009e7b665eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:14.665337 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c149520-eef2-47a6-a360-009e7b665eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:14.666736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c149520-eef2-47a6-a360-009e7b665eab service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:14.676793 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1429/2851] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:13:14.956701 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:14.958192 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:14.972485 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e868d-311c-4457-b26a-f46cacf98a0e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:14.978026 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:14.978902 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:14.979183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:14.979489 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] PATH_INFO: `/v3/projects/85c01920348f4ebeb98c5ea2f3ddde93` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:14.980738 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c01920348f4ebeb98c5ea2f3ddde93)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:14.982678 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.040357 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:15.041667 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c01920348f4ebeb98c5ea2f3ddde93', 'request_id': 'req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:15.049773 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c01920348f4ebeb98c5ea2f3ddde93', 'request_id': 'req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:15.058183 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c01920348f4ebeb98c5ea2f3ddde93', 'request_id': 'req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:15.070153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] The token cache is being invalidate because project 85c01920348f4ebeb98c5ea2f3ddde93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:13:15.070447 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:15.071083 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5d013c11-53b9-41c2-aeeb-5aa7d6b8e5c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:15.072710 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1423/2852] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:13:14 2022] DELETE /identity/v3/projects/85c01920348f4ebeb98c5ea2f3ddde93 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:15.077948 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.078538 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.078824 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.099047 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:15.100947 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:15.164840 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1430/2853] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:15.173480 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.173772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.174056 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.196609 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:15.198117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:15.207100 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1424/2854] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:13:15.212055 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.222094 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c5ae6-424c-403f-9a50-b60fbc17bf90 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.227331 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.228004 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.228285 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.228596 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.229332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:13:15.229636 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.231257 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.246170 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1431/2855] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:13:15.250590 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.261272 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00b3bd-2174-44ca-b97b-15b07b7d7093 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.266041 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.266709 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.266993 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.267274 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.267971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.269936 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.300798 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1425/2856] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:13:15 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:13:15.305267 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.315592 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a1638-1186-4ea7-98f6-3701ad83bd9b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.374552 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.375230 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.375498 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.375772 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.376509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.378419 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.388121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:15.402085 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1432/2857] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:13:15 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:13:15.406337 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.418790 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea3ff7-4bc1-49c2-9c61-ee67ca79f2b4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.424376 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.425148 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.425464 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.425781 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.426561 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.428229 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.439450 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1426/2858] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 14:13:15.443453 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.454530 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9406c-b29d-4154-801f-a1d6679f7ca6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.459318 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefe4ff-83a9-4657-b3f4-c0286438a4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.460524 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe4ff-83a9-4657-b3f4-c0286438a4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.460808 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe4ff-83a9-4657-b3f4-c0286438a4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.461082 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefe4ff-83a9-4657-b3f4-c0286438a4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.461815 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe4ff-83a9-4657-b3f4-c0286438a4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.463416 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefe4ff-83a9-4657-b3f4-c0286438a4fb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.474107 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1433/2859] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:13:15.478579 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.490442 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a12080-0499-42c1-9073-24eb0e37c931 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.495396 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.496317 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.496640 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.496920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] PATH_INFO: `/v3/projects/871c1fd7e93a4a14a62807240c3ad68f/users/4c14c12aa0c94275a567f3bd51b6ac6b/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.513598 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871c1fd7e93a4a14a62807240c3ad68f, user_id=4c14c12aa0c94275a567f3bd51b6ac6b, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.516286 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.531750 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1427/2860] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:13:15 2022] PUT /identity/v3/projects/871c1fd7e93a4a14a62807240c3ad68f/users/4c14c12aa0c94275a567f3bd51b6ac6b/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:15.601304 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b336c-f38d-48d1-ac67-4df15346ec83 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.601652 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b336c-f38d-48d1-ac67-4df15346ec83 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.601977 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b336c-f38d-48d1-ac67-4df15346ec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.633160 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-693b336c-f38d-48d1-ac67-4df15346ec83 None None] MFA Rules not processed for user `4c14c12aa0c94275a567f3bd51b6ac6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:15.634464 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-693b336c-f38d-48d1-ac67-4df15346ec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:15.706372 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1434/2861] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:15.718650 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:15.720440 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:15.783992 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff57d8b-5340-4647-8846-dbb8a525fb03 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:15.788060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:15.788783 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:15.789077 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:15.789354 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:15.799846 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:15.801367 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:15.812455 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1428/2862] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:13:25.703103 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:25.807567 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=60,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:53.888264 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:53.888846 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:53.888961 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:53.906569 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] MFA Rules not processed for user `1e956289b0e64cc892d453f427845d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:53.907708 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:53.961971 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1435/2863] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:54.287933 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:54.289649 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:54.351518 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71498a-74e4-44eb-8564-c70cb7ef5d5e service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:54.355130 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:54.355777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:54.356061 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:54.356343 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:54.358266 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:54.368282 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:54.369751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda35a4-2e89-492a-b396-8821e5fbae7d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:54.385189 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1429/2864] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:13:57.893347 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:57.970378 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e3a9a-1d75-4e3d-81b4-5da318ee8153 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:57.975012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:57.975891 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:57.976205 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:57.976518 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] PATH_INFO: `/v3/users/4c14c12aa0c94275a567f3bd51b6ac6b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:57.977750 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c14c12aa0c94275a567f3bd51b6ac6b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:57.979683 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:58.033232 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c14c12aa0c94275a567f3bd51b6ac6b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.048901 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c14c12aa0c94275a567f3bd51b6ac6b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.057408 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c14c12aa0c94275a567f3bd51b6ac6b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.058088 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c14c12aa0c94275a567f3bd51b6ac6b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.058704 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c14c12aa0c94275a567f3bd51b6ac6b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.072298 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1436/2865] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:13:57 2022] DELETE /identity/v3/users/4c14c12aa0c94275a567f3bd51b6ac6b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:58.077259 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:58.077880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:58.077880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:58.096983 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:58.098243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:58.168467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1430/2866] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:58.181623 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:58.183114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:58.214604 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:58.256736 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4865b-3955-404b-b532-110fdf2b72b3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:58.260513 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007d6f0-cbd3-4820-a23c-b88e7b43679a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:58.261186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007d6f0-cbd3-4820-a23c-b88e7b43679a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:58.261526 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007d6f0-cbd3-4820-a23c-b88e7b43679a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:58.261756 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007d6f0-cbd3-4820-a23c-b88e7b43679a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:58.272599 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007d6f0-cbd3-4820-a23c-b88e7b43679a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:58.274062 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007d6f0-cbd3-4820-a23c-b88e7b43679a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:58.284855 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1437/2867] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:13:58.785547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:58.787509 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:58.800155 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad847f-bde1-49f2-af10-3d753bde70e8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:58.805628 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:58.806496 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:58.806772 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:58.807044 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] PATH_INFO: `/v3/projects/871c1fd7e93a4a14a62807240c3ad68f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:58.808230 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] RBAC: Authorizing `identity:delete_project(project_id=871c1fd7e93a4a14a62807240c3ad68f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:58.810254 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:58.875880 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871c1fd7e93a4a14a62807240c3ad68f', 'request_id': 'req-e118ab39-9547-46e5-b953-47a73fc5650f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.884435 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871c1fd7e93a4a14a62807240c3ad68f', 'request_id': 'req-e118ab39-9547-46e5-b953-47a73fc5650f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.892838 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871c1fd7e93a4a14a62807240c3ad68f', 'request_id': 'req-e118ab39-9547-46e5-b953-47a73fc5650f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.905815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] The token cache is being invalidate because project 871c1fd7e93a4a14a62807240c3ad68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:13:58.906108 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.906786 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e118ab39-9547-46e5-b953-47a73fc5650f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:13:58.908371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1431/2868] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:13:58 2022] DELETE /identity/v3/projects/871c1fd7e93a4a14a62807240c3ad68f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 14:13:58.914639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:58.914639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:58.914639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:58.933613 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:58.934997 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:58.999307 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1438/2869] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:59.011891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.012180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.012490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.031039 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:59.032256 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:59.040971 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1432/2870] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:59.045434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.056508 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09875f9-6634-4c77-b065-b800264ef133 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.061332 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.062087 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.062444 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.062795 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.063608 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 14:13:59.063990 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.065769 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.078966 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1439/2871] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jul 15 14:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 14:13:59.084244 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.093843 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602de2cb-eb9d-4a34-b0bf-84b40182a800 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.098676 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.099900 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.100181 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.100490 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.101194 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.103219 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.124277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:59.132780 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1433/2872] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jul 15 14:13:59 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 14:13:59.137346 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.149658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e464cbf-1f8e-4788-9f6a-84239e475e03 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.206778 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.207446 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.207728 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.208010 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.209223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.211121 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.232172 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1440/2873] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jul 15 14:13:59 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 14:13:59.236504 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.247047 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a6560-315c-4ef6-b2a6-e8f63378f1c5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.251776 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.252449 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.252734 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.253010 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.253741 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.255327 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.265668 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1434/2874] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:13:59.272178 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.285814 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f4265-017f-4238-a514-17f5b703e479 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.290959 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228fdc2-150c-41b6-9ce4-56f1daa75ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.291637 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228fdc2-150c-41b6-9ce4-56f1daa75ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.291911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228fdc2-150c-41b6-9ce4-56f1daa75ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.292188 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228fdc2-150c-41b6-9ce4-56f1daa75ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.292951 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228fdc2-150c-41b6-9ce4-56f1daa75ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.294532 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228fdc2-150c-41b6-9ce4-56f1daa75ab1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.304397 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1441/2875] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jul 15 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 14:13:59.308591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.318409 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c895699-26b6-44d6-8535-e53fc487dbbb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.322918 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.323791 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.324069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.324344 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] PATH_INFO: `/v3/projects/3b77053c420a4d1b83cf14e7d8129755/users/7e5ec9eafc434dbab75eef7288cc1a85/roles/1773094ced584f8fbdccdb7ce4502fe4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.338953 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b77053c420a4d1b83cf14e7d8129755, user_id=7e5ec9eafc434dbab75eef7288cc1a85, role_id=1773094ced584f8fbdccdb7ce4502fe4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.341610 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.354823 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1435/2876] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jul 15 14:13:59 2022] PUT /identity/v3/projects/3b77053c420a4d1b83cf14e7d8129755/users/7e5ec9eafc434dbab75eef7288cc1a85/roles/1773094ced584f8fbdccdb7ce4502fe4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:13:59.360862 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf62c-b80d-4f7c-97b9-3e6ff4dc55bd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.361162 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf62c-b80d-4f7c-97b9-3e6ff4dc55bd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.361439 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cf62c-b80d-4f7c-97b9-3e6ff4dc55bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.381013 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-404cf62c-b80d-4f7c-97b9-3e6ff4dc55bd None None] MFA Rules not processed for user `0802dbaa2668431bbb6defab89b49e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:13:59.382242 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-404cf62c-b80d-4f7c-97b9-3e6ff4dc55bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:59.410425 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:13:59.442721 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1442/2877] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:13:59.459166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:13:59.459166 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:13:59.520851 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e479f87-687c-41ef-9492-0fb833119503 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:13:59.525257 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:13:59.525920 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:13:59.526195 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:13:59.526477 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:13:59.536578 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:13:59.538053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:13:59.548522 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1436/2878] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:14:00.176415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a58fc-43e5-4246-88e9-817692ae4a8f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:14:00.176929 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a58fc-43e5-4246-88e9-817692ae4a8f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:14:00.177884 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a58fc-43e5-4246-88e9-817692ae4a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:14:00.215359 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-891a58fc-43e5-4246-88e9-817692ae4a8f None None] MFA Rules not processed for user `7e5ec9eafc434dbab75eef7288cc1a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:14:00.216639 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-891a58fc-43e5-4246-88e9-817692ae4a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:00.299584 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1443/2879] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:14:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:14:00.310841 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:14:00.312349 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:00.384029 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419ac54-ef78-4223-8453-e93596ad8666 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:14:00.387904 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0e94eecf28f42468e14bbd0c4eb3777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:14:00.388731 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:14:00.389017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:14:00.389293 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:14:00.399149 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:14:00.400769 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:14:00.410540 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1437/2880] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:14:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:14:01.771305 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:14:01.772129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:14:01.772129 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:14:01.790486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] MFA Rules not processed for user `7e5ec9eafc434dbab75eef7288cc1a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:14:01.791799 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:01.799643 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1444/2881] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 14:14:03.247855 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:14:03.249599 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:03.326289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b91ec-ca1c-4e27-9989-fc0e80e4a43f service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:14:03.331624 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:14:03.332289 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:14:03.332601 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:14:03.332884 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:14:03.334718 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:03.347180 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:14:03.348807 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:14:03.353754 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:14:03.364017 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1438/2882] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:14:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:14:05.996355 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:14:05.998965 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:06.012015 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d838e6-7867-47ad-bdcd-42f1eb598dd3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:14:06.016222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:14:06.017349 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:14:06.017701 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:14:06.018114 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:14:06.020243 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:06.093709 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:14:06.095577 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd511c7-e415-4359-a51c-fa0a23e91a8e service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:14:06.107234 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1445/2883] 10.222.0.16 () {60 vars in 1396 bytes} [Fri Jul 15 14:14:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:14:12.458891 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:14:12.460462 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:12.560060 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a3c23-af53-4d30-a20b-b10ef021fc2d service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:14:12.566324 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e424ea6d03414857bc408e49b8df5429', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:14:12.566535 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:14:12.566821 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:14:12.567098 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:14:12.569217 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:14:12.590815 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:14:12.592457 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:14:12.606371 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1439/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:14:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:14:16.102258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=75 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:14:22.601167 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=35 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:15:07.500289 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:15:07.500984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:15:07.500984 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:15:07.519007 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] MFA Rules not processed for user `7e5ec9eafc434dbab75eef7288cc1a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 14:15:07.520222 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:15:07.527316 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1446/2885] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jul 15 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 14:15:07.758007 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:15:07.767468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabf412-32a2-4561-a63f-8de6ac3a4af7 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:15:07.771127 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:15:07.771873 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:15:07.772153 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:15:07.772468 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:15:07.775197 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:15:07.786827 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:15:07.788278 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:15:07.797735 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1440/2886] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:15:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:15:17.513952 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:15:17.793179 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:16:00.844223 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:00.854454 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548574a5-41ff-4477-bcbb-e4dd19b97093 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:00.858038 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '393c4bea16074727958ea40ab497948d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:00.858682 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:00.858969 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:00.859268 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:00.861109 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:00.874658 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:00.876161 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:00.886064 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1447/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 14:16:03.740704 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:03.742326 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:03.763069 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a74d6-ed7c-47ca-98e5-26d14189d286 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:03.769737 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:03.770467 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:03.770751 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:03.771277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:03.773727 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:03.788015 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:03.789647 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:03.817903 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1441/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:16:07.142058 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:07.143754 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:07.158141 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e79e-1e80-4967-a4b3-60718cebd4b4 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:07.168176 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:07.169181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:07.169466 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:07.169773 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] PATH_INFO: `/v3/users/1e956289b0e64cc892d453f427845d71` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:07.171005 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e956289b0e64cc892d453f427845d71)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:07.173037 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:07.220415 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e956289b0e64cc892d453f427845d71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.232718 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e956289b0e64cc892d453f427845d71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.241258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e956289b0e64cc892d453f427845d71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.241878 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e956289b0e64cc892d453f427845d71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.242509 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e956289b0e64cc892d453f427845d71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.255118 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1448/2889] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:16:07 2022] DELETE /identity/v3/users/1e956289b0e64cc892d453f427845d71 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:16:07.438453 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:07.450117 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee04441-78d9-4199-be6d-6711645844ad service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:07.510187 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:07.511053 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:07.511336 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:07.511615 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] PATH_INFO: `/v3/projects/ddbd6e1cb0814523b4e528c14d06cdad` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:07.512867 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbd6e1cb0814523b4e528c14d06cdad)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:07.514777 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:07.566203 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbd6e1cb0814523b4e528c14d06cdad', 'request_id': 'req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.572277 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbd6e1cb0814523b4e528c14d06cdad', 'request_id': 'req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.579373 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbd6e1cb0814523b4e528c14d06cdad', 'request_id': 'req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.590204 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] The token cache is being invalidate because project ddbd6e1cb0814523b4e528c14d06cdad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:16:07.590486 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.591235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.592964 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1442/2890] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:16:07 2022] DELETE /identity/v3/projects/ddbd6e1cb0814523b4e528c14d06cdad => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:16:07.597006 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:07.598434 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:07.611579 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389cdf1-9d04-4861-a0a1-9471a0385098 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:07.667700 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:07.668598 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:07.668881 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:07.669153 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] PATH_INFO: `/v3/users/a003de4e882049479c9aa10b36495ee9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:07.670353 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a003de4e882049479c9aa10b36495ee9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:07.672259 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:07.718136 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a003de4e882049479c9aa10b36495ee9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.729255 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:16:07.731917 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a003de4e882049479c9aa10b36495ee9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.741554 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a003de4e882049479c9aa10b36495ee9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.742523 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a003de4e882049479c9aa10b36495ee9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.743167 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a003de4e882049479c9aa10b36495ee9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:07.757181 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1449/2891] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:16:07 2022] DELETE /identity/v3/users/a003de4e882049479c9aa10b36495ee9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:16:07.947872 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:07.949637 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:07.965712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a6c77-c57c-4cd0-8e87-c51ca5c1afa8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:08.034419 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:08.035330 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:08.035663 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:08.036031 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] PATH_INFO: `/v3/projects/926204d660974b039dfb2d21b87d8d7d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:08.037307 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926204d660974b039dfb2d21b87d8d7d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:08.039225 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:08.061712 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:16:08.094614 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926204d660974b039dfb2d21b87d8d7d', 'request_id': 'req-999978ab-0573-4efb-b5e1-ff4ca44a8394'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.101113 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926204d660974b039dfb2d21b87d8d7d', 'request_id': 'req-999978ab-0573-4efb-b5e1-ff4ca44a8394'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.109071 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926204d660974b039dfb2d21b87d8d7d', 'request_id': 'req-999978ab-0573-4efb-b5e1-ff4ca44a8394'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.122302 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] The token cache is being invalidate because project 926204d660974b039dfb2d21b87d8d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:16:08.122591 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.123251 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.124864 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1443/2892] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:16:07 2022] DELETE /identity/v3/projects/926204d660974b039dfb2d21b87d8d7d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:16:08.631433 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:08.633012 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:08.647320 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5339c7a-1753-4d22-98dc-0bf09e1d24d6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:08.703992 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:08.704911 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:08.705196 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:08.705474 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] PATH_INFO: `/v3/users/7e5ec9eafc434dbab75eef7288cc1a85` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:08.707258 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5ec9eafc434dbab75eef7288cc1a85)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:08.709202 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:08.755684 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5ec9eafc434dbab75eef7288cc1a85'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.769374 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5ec9eafc434dbab75eef7288cc1a85'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.776841 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5ec9eafc434dbab75eef7288cc1a85'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.777539 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5ec9eafc434dbab75eef7288cc1a85'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.778503 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5ec9eafc434dbab75eef7288cc1a85'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:08.791412 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1450/2893] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jul 15 14:16:08 2022] DELETE /identity/v3/users/7e5ec9eafc434dbab75eef7288cc1a85 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:16:08.970090 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:16:08.971730 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:16:08.987547 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999978ab-0573-4efb-b5e1-ff4ca44a8394 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:16:09.045394 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0802dbaa2668431bbb6defab89b49e81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8618b94bef7144e799c6d51bb1ef841f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:16:09.046243 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:16:09.046525 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:16:09.046800 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] PATH_INFO: `/v3/projects/3b77053c420a4d1b83cf14e7d8129755` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:16:09.047976 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b77053c420a4d1b83cf14e7d8129755)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:16:09.050423 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:16:09.096801 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b77053c420a4d1b83cf14e7d8129755', 'request_id': 'req-25173032-03e5-41cb-84f2-dd1f38c75ddd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:09.102929 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b77053c420a4d1b83cf14e7d8129755', 'request_id': 'req-25173032-03e5-41cb-84f2-dd1f38c75ddd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:09.111471 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b77053c420a4d1b83cf14e7d8129755', 'request_id': 'req-25173032-03e5-41cb-84f2-dd1f38c75ddd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:09.123688 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] The token cache is being invalidate because project 3b77053c420a4d1b83cf14e7d8129755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 14:16:09.123980 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:09.124659 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-25173032-03e5-41cb-84f2-dd1f38c75ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 14:16:09.126235 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1444/2894] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jul 15 14:16:08 2022] DELETE /identity/v3/projects/3b77053c420a4d1b83cf14e7d8129755 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 14:16:18.787874 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=12,keystone:SELECT=60,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:16:19.120569 dv-cinder-850002-1-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 14:19:11.807186 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 14:19:11.808657 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:19:11.951291 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1898b7d-8a1d-4106-8afc-429a69e1b088 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 14:19:12.010486 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92ac8ddf274a45d1b56533944a2334fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3daa91b4a48b4ed0bc5f2cc3b87094ae', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 14:19:12.011166 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 14:19:12.011449 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 14:19:12.011726 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 14:19:12.013556 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 14:19:12.025338 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 14:19:12.026915 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197e583-d044-4352-bd47-1ca814a40bfa service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 14:19:12.036912 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1451/2895] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jul 15 14:19:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 14:19:22.032602 dv-cinder-850002-1-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}